Bittensor Subnet 31:Naschain

Network Architecture Search (NAS) is essential when designing customized neural network architectures to meet specific computational constraints and accuracy requirements.NAS involves training numerous models through an iterative process to find optimal architectures tailored to the given criteria.

Genetic Algorithm-Based NAS: NASChain utilizes a genetic algorithm to optimize neural networks, representing each network as a binary-encoded “genome.” This method allows for a systematic exploration of architectural possibilities. The process begins by submitting labeled data sets and possibly a labeled test set to initiate training and evaluation. The Genomaster orchestrates the training tasks among the miners, utilizing a genetic algorithm to create diverse neural network architectures. Miners receive unique architectures represented in binary code, train them based on set parameters, and provide feedback on their performance. Each architecture’s fitness, evaluated by factors such as accuracy, memory requirements, and floating operations, determines their success.

Optimization Process: After each generation of training models, the orchestrator evaluates, ranks, and mutates the top-performing architectures for further improvement. This iterative process continues through multiple generations, gradually refining the architectures towards optimal solutions while considering trade-offs like accuracy versus memory efficiency. Through approximately 50 generations of mutations and crossovers, the algorithm converges on a set of dominant genome solutions that offer the best trade-offs according to the user’s preferences. The optimization process culminates in presenting the user with a selection of optimal models based on multi-objective optimization criteria. Users can select from various choices that balance factors like accuracy and memory efficiency, offering flexibility in choosing models that best suit their specific needs. By providing insights into the trade-offs between different architectures, users can make informed decisions based on their priorities, whether focused on accuracy or resource optimization.

Distributed Training: By leveraging the Bittensor network, NASChain decentralizes the intensive computational process, enabling parallel genome training by a network of miners. Miners’ role is to receive, train, and respond within the architecture orchestrated by the Central system. Challenges include validating the training of miners, tackling miner connectivity issues, and ensuring training accuracy. Convergence is reached when improvement across generations declines, indicating no further gain in accuracy or reduction in computing requirements. Users can monitor convergence through live plotting, observing a reduction in size and genome adjustments.

Blockchain Integration: This ensures security and transparency, with miners rewarded for contributing computational resources towards training and evaluating network models.

Outcome: The process yields optimal neural architectures that balance high accuracy with low computational demands, achieved more efficiently through distributed efforts. The algorithm in NASChain employs the NSGA approach for optimization.

What is the outcome of a NAS experiment?

Once the NAS run is completed for a specific use case (e.g., a dataset for classification), a visualization tool and a post-processing script can extract dominant genomes from the list of all genomes trained across generations. Given that NAS is a multi-objective optimization problem, there will be more than one optimal solution. These dominant genomes form a Pareto optimal frontier nomes on this frontier are considered the most optimal architectures for the use case, balancing accuracy, number of parameters, or FLOPs.

The system can be adapted for time series tasks and regression analysis, expanding its utility beyond image classification. The architecture is customizable for different use cases, such as detecting stock prices and other regression tasks.

Hardware Requirements

Miners: GPU: Nvidia GPU with at least 16GB of memory. Note that 8GB graphics cards might work in some cases, but their compatibility and performance are not guaranteed.

Validators: CPU: Machines with only CPUs are sufficient for validators as they do not undergo intensive computational loads.

Two-Level Validation and Incentive System:

• Subnet 31 incorporates a dual validation and incentive system, rewarding miners for proof of work and productivity based on job completion times. Miners submit responses to the Genomaster in arrays [accuracy, parameters, FLOPs]. Each job is randomly assigned to three miners to ensure legitimacy.

Level 1: Agreement-Based PoW Scoring

1. Job Batches: Defined as B = {b_1, b_2, …, b_n}, each batch corresponds to evaluations from different users for the same task.
2. Distribution: Each unique genome training task is randomly assigned to three miners to avoid duplication.
3. Responses: Miners submit results [M_A, M_P, M_F] representing accuracy, parameters, and FLOPs.
4. Consensus: Validators assess results in a 3×3 matrix, requiring accuracy within ±1% and exact agreement on parameters and FLOPs.
5. Reward Allocation: Miners receive points if all three agree; if two agree, those miners are rewarded; disagreement results in no points.

Level 2: Total Jobs Completed

• Miner productivity is evaluated based on total jobs completed, incentivizing both accuracy and speed. Faster GPUs contribute more jobs, enhancing rewards and encouraging adherence to configurations for reliability.

This system promotes productivity and ongoing improvement, rewarding efficient miners and maintaining accuracy and consistency in computational tasks.