Bittensor Subnet 22: Desearch

For Subnet 22, the vision is to incentivize miners who excel in scraping and indexing multiple databases, retrieving that data quickly, and using a language model to accurately summarize raw data. This allows us to effectively respond to user queries. To achieve this, miners must have APIs to various sources, including Twitter, Reddit, Google, YouTube, and more. Currently, the primary focus is on perfecting Twitter relevance scores before moving on to other databases. The reward mechanism heavily prioritizes Twitter data relevancy.

Here’s how it works: A query is sent to miners, such as “What is augmented reality’s daily impact in 2024?” Miners use APIs or indexing tools to search their databases and find the best tweets to answer the question. Typically, this involves multiple complex Twitter API calls crafted by trained language models. Miners then return up to 10 tweets in response. Once the validator receives these tweets, they use a language model with context to score them according to specific criteria, which can be found here.

Overview of the Subnet Scoring System

The subnet scoring system evaluates the performance of miners across three key metrics: Summary Scoring, Twitter Scoring, and Search Scoring. This approach ensures that responses are not only relevant but also insightful and comprehensive. The scoring system involves connecting to “subnet 18” to leverage its AI models and cloud capabilities, allowing for a more comprehensive evaluation of minor responses. Here’s how each component is structured:

1. Summary Scoring (40%)
   • Assesses the quality of the summary provided by the miner after selecting relevant Twitter links.
   • Emphasizes the relevance of the summary to the original prompt, depth and comprehensiveness of the information, and clarity and precision of the summary.
2. Twitter Scoring (50%)
   • Makes up half of the total score, requiring miners to submit 10 Twitter links closely related to the query.
   • Top performers conduct thorough searches using the Twitter API to find the most relevant and informative tweets, ensuring they directly address specific aspects of the prompt with depth and valuable insights.
3. Search Scoring (10%)
   • Evaluates the relevance of web links provided by miners in response to the initial prompt.
   • Effective responses should mention and engage with keywords and themes from the prompt, providing insightful and substantial engagement with the topics.

Effective Scoring Strategy

The scoring system is designed to motivate miners to deliver responses that are not just relevant but also rich in content and insight. By excelling in each area, miners can significantly improve their overall score and, consequently, their rewards. This structured scoring approach ensures consistency and fairness while motivating ongoing improvement in response quality.

The best way to improve a miner’s score is to enhance the relevance score by improving the quality of API calls to Twitter.

On Distribution and Quality

Regarding distribution, it doesn’t matter if only one miner achieves high quality. If one miner can provide the best results, they deserve all the rewards. The focus is on quality, not decentralization for its own sake. Decentralization means equality of opportunity, which is already present as everyone has access to the same information.

The subnet is open-source, and all validator logs are exposed through Weights and Biases here, including all miner scores and associated content. Transparency in how to mine and improve scores is maintained, and community feedback is taken into account. The goal is to have the best miners, regardless of quantity.

Addressing Miner Complaints

Complaints from miners who struggle are often due to a lack of research. The process is clear with open-source code, and miners should compare their answers to others using tools like Weights and Biases, testing various strategies for Twitter API calls. If their methods aren’t as effective as the top miners, deregistration is necessary to maintain high-quality responses.

Deregistering miners who were previously doing well indicates continuous improvement within the subnet. Constant competition ensures only the best miners remain, driving overall subnet quality.

Enhancing Cost Efficiency and Participation

Utilizing Subnet 18’s capabilities reduces the costs associated with running models independently by providing a more cost-effective alternative for validators and miners. Leveraging Subnet 18 not only lowers operational expenses but also broadens the network’s capabilities by utilizing existing resources efficiently, benefiting all participants. Miners can utilize the resources of subnet 18 for scoring, creating a collaborative environment that rewards both contributions and participation.

Key Features

• AI-Powered Analysis: Uses artificial intelligence to provide deeper insights into user interactions on Twitter.
• Real-Time Data Access: Connects directly to Twitter’s database for the most up-to-date information.
• Sentiment Analysis: Assesses the emotional tone of tweets to understand public sentiment.
• Metadata Analysis: Examines tweet details such as timestamps and retweet counts for a thorough overview.
• Time-Efficient: Reduces manual data sorting, saving valuable research time.
• User-Friendly Design: Accessible for both beginners and experts.

Advantages

• Decentralized Platform: Ensures reliability by operating on the Bittensor network.
• Customizability: Adapts data analysis to meet specific user needs.
• Informed Decision-Making: Supports data-driven strategies.
• Versatility: Suitable for various research fields, from market analysis to academic studies.

Subnet 22 aims to incentivize miners to provide high-quality data retrieval and summarization services by leveraging APIs and indexing tools. The focus on relevance scores and the use of language models to validate results ensure that the best miners are rewarded for their efforts. While some may argue that decentralization is compromised when a single entity dominates the mining landscape, what matters most is the quality of the product and the equality of opportunity for all participants.