Submit
Please follow the directions below to submit to the red team exercise.
Thank you for joining us and helping us learn!
Quick Links (this page):
Participation
What Will I Submit?
This program accepts attack predictions for challenge records in CSV format ( see the problem statement for an example). We'll provide an accuracy score on your predictions at leaderboard updates. You never submit any code to us.What Will I Get If I Submit?
You'll see your accuracy score and self-selected team name on our , unless you choose to submit anonymously.How to Submit:
Step 1:
Sign up for our listserv (submit an empty email to subscribe). We use the listserv for program updates only.Step 2:
Register to participate to receive your accuracy score and receive recognition on the leaderboard. By default, we only publish your self-chosen team name and your accuracy score. You can participate as a single player, in a team of any number, or on multiple teams.Step 3:
Phase 1 Soybean Data ("Don't Spill the Beans!") March - July:- Problem 1: Membership Inference Attack:
- Read the Problem Statement
- Download the Competitor's Pack
- Read the Problem Statement
- Download the Competitor's Pack
Phase 2 Dog Data: June- July
- Problem 3: Membership Inference Attack
- Read the Problem Statement
- Download the Competitor's Pack
- Problem 4: Reconstruction Attack
- Read the Problem Statement
- Download the Competitor's Pack
Step 4:
Follow the directions in the competitor's pack to prepare your submission- Check out our resources page for helpful libraries and research.
- Note you can submit up to three attempts for every leaderboard update deadline, using different approaches/configurations. Your team will be scored by your highest performing entry. This gives you a chance to try out multiple libraries or ideas.
Step 5:
Submit using the correct form for the problem. Check the Calendar in the Problem Statements page for leaderboard update submission deadlines. Each problem has multiple opportunities to submit and see the scores of your current approaches.Phase 1 Soybean Data ("Don't Spill the Beans!") March - July:
- Problem 1: Membership Inference Attack. Submit here.
- Problem 2: Reconstruction Attack: Submit here.
Phase 2 Dog Data: June - July
- Problem 3: Membership Inference Attack Submit here.
- Problem 4: Reconstruction Attack Submit here.
Some Notes on the Problem Structure
Teams are welcome to register at any point during the exercise and submit an entry for any problem. Participating in Phase 1 isn't required for participating in Phase 2.We give you multiple opportunities to submit to the leaderboard for each problem, to help you learn which approaches are more successful. Each leaderboard update provides you with a high level total accuracy percentage on each of the privacy levels you attacked.
To support participants who want to experiment with different libraries and approaches, you can submit up to three attempts for each leaderboard update.
In each problem we'll provide you with some demonstration data; essentially a part of the problem where we give you the answers for free, so you can try out different techniques locally before you submit. In the Phase I Problem 1 the competitor's pack includes the membership list for Client 4. In Phase I Problem 2 we provide answers for Client 2.
If you have any questions at any time, please reach out to us by email, or on the #crc-nist-office-hours channel of the OpenDP Slack (Connect).
Join us for office hours every Wednesday at 11am ET.