The project was developed with Vasiliki Batsari, Kiarash Rezaei and Francesco Ruan in collaboration with Micron.
The problem formulation and the assistance for the project were provided by Micron within the initiative Adopt a course between the companies and Politecnico di Milano.
The goal of the project is to predict the efficiency of Polar Codes using Deep Neural Networks. Based on the trained model, the objective is to leverage its parameters to produce a configuration of the polar codes with high efficiency. The evaluation the the efficiency is carried out by analyzing the performance of the constellation for a different set of Signal to Noise Ration (SNR). For further details on polar codes refer to this paper.
The results are simulated for polar codes with 64 information bytes. We also provide the description of data simulation for other configurations using Aff3ct Toolbox. Moreover, we provide the search of the optimal configuration of bits using Random Search and Projected Gradient Descent.
To generate the initial dataset, we utilized Aff3ct. This simulator provides a Frame Error Rate (FER) and Bit Error Rate (BER) for a sequence of frozen and information bits' positions and SNR using Monte Carlo (MC) simulation.
We started from a frozen set of bits for Gaussian Approximation (GA) we applied random permutations to the bits to get the required number of samples (~15K samples of 1024 bits/64 bytes of payload). We automated the data acquisition process using bash scripting. The results are saved in two txt files, where fb represents the code structure and fer - the corresponding metric.
When the task is to generate the code configuration, we save the proposed results into txt files with rnd and pgd being the identifiers of the generating algorithm.
First, we discard the bits with zero variance along the sample dimensions, as these features are uninformative. We further standardize the bits by converting them to +-1, while the FER is converted to linear units. The converter also capable of converting the data backwards by including the redundant bits and leveraging stored means per bit, and taking a proper logarithm from the labels. Later the data is converted to a torch dataset and batched using DataLoader.
We simulate the multi-layer perceptron (MLP)-based architecture. The controllable parameters are the following:
- Number of Training Epochs
- Hidden Dimension Size
- Depth of The Network
- Frequency of Skip-connections
The model is evaluated in terms of Inflation of Error (IOE), which indicates the percentage of how inaccurate the output is with respect to the true value.
We provide a broad analysis of how the parameters impact the performance of the network on the final output. The figures can be found in the report.
In order to predict an optimal frozen bit set two following methods were used: Projected Gradient Descent and Random Search algorithms. The goal for both of them is to find the input such that the output of the model is maximized.
Random Search (RS) at each time step inputs 1000 random string of bits (maintaining R = ½) and produces corresponding FERs. In case the model predicts the FER is smaller than the best sample in the dataset, the input is stored in the file. This process is repeated for a fixed amount of time.
Projected Gradient Descent (PGD) with frozen network parameters attempts to optimize the input by minimizing the output using Stochastic Gradient Descent (SGD) and projecting the input on the discrete. The procedure is repeated for a number of iterations, after which the input configurations better than the best in the dataset (if any) are stored.
The final results are presented in this file. The results were presented online in front of Micron's engineers and received the maximum grade for the project.
Léonardon, Mathieu & Gripon, Vincent. (2021). Using Deep Neural Networks to Predict and Improve the Performance of Polar Codes. 1-5. 10.
Nachmani, Eliya & Marciano, Elad & Lugosch, Loren & Gross, Warren & Burshtein, David & Be'ery, Yair. (2017). Deep Learning Methods for Improved Decoding of Linear Codes. IEEE Journal of Selected Topics in Signal Processing. 12. 10.
Huang, Lingchen & Zhang, Huazi & Li, Rong & Ge, Yiqun & Wang, Jun. (2019). AI Coding: Learning to Construct Error Correction Codes.