Experiment *2: Olympus DAO Generating Solutions

MetricsDAO OODAD Experiments
1

Goal: Enable and incentivize Analysts to solve Challenges.

MetricsDAO provides a 6 step process for ​​Organized, On-Demand Analytics Delivery, or as we like to call it, OODAD.

After curating the blockchain data, sourcing questions, and turning them into Challenges for Analysts, the fourth step is enabling the community to provide results, visualizations, insights etc. to address these specific Challenges. We refer to this as generating “Solutions”.

This is where our second experiment comes in.

After the community documented and prioritized needs for OlympusDAO via a question bank, analysts were offered bounties for providing Solutions.

Setting Up Bounties: Questions to Consider

Setting up these bounties in a way that would successfully enable and incentivize analysts to solve these Challenges raised important questions:

  1. Due Date -→ How long should Analysts have to complete a specific challenge? -→ Who determines how long a challenge should take to complete?
  2. Payout -→ How do we determine the right payout per challenge? -→ Should the payout amount be stated ahead of time, or after completion once the quality of the submission has been determined?
  3. Driving Engagement -→ How do we get more Analysts to generate solutions for these challenges?
  4. Transparency -→ Do we publicly display solutions before they’ve been reviewed?

Initial Set Up

Here’s how we chose to answer these questions in the first experiment:

  1. Due Date The bounties were initially due on December 13th 2021, which gave Analysts 7 days to complete the Challenges. A week seemed reasonable based on the difficulty of the questions asked, but we did not ask Analysts for input here.
  2. Payout The payout was set at $250 for the top 3 Solutions. The next 22 Solutions would all receive $100. These were to be paid out in OHM. The payout amount was announced ahead of time but did not vary based on question difficulty.
  3. Driving Engagement We promoted these bounties on Twitter and Discord but sought no cross-promotion from OlympusDAO.
  4. Transparency We decided to not share submissions before reviewing them.

Outcomes

  1. Due Date The turnaround was too short and Analysts asked for more time. We extended the due date by 48 hours and received 4 more submissions, for a total of 11 submissions.
  2. Payout Dollar amount was deemed too low, especially considering the high transaction fees on the Ethereum network. The flat payout (especially of $100) was deemed too low for how hard the questions were. We also found out that Analysts couldn’t easily tell which questions were hard vs. easy before starting to work on them, and they understandably did not want to answer questions without knowing ahead of time how hard they would be.
  3. Driving Engagement We received 11 submissions for 10 challenges. Three Challenges were not answered.
  4. Transparency Community members wanted to know when they would be able to see the submitted Solutions.

Lessons Learned

  1. Due Date Difficulty should be determined ahead of time, so that due dates are properly aligned.

  2. Payout Payment needs to be tailored to question difficulty, and Analysts need to know how hard a Challenge is before starting. Payment date should be announced ahead of time. We will experiment with adding a step in the process where the community determines how much each Challenge is worth, based on difficulty.

  3. Driving Engagement We should work with the crypto project itself to help us cross-promote new bounties to their community.

  4. Transparency We should announce ahead of time: -→ when POAPs will go out-→ when Analysts should expect to get paid and -→ when submissions will be made public.