> ## Documentation Index
> Fetch the complete documentation index at: https://nixtla-old-docs.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

> Download and evaluate the M5 dataset.

# M5 dataset

## Download data class

***

<a href="https://github.com/Nixtla/datasetsforecast/blob/main/datasetsforecast/m5.py#L18" target="_blank" style={{ float: "right", fontSize: "smaller" }}>source</a>

### M5

> ```text theme={null}
>  M5 (source_url:str='https://github.com/Nixtla/m5-
>      forecasts/raw/main/datasets/m5.zip')
> ```

## Test number of series

## Evaluation class

***

<a href="https://github.com/Nixtla/datasetsforecast/blob/main/datasetsforecast/m5.py#L152" target="_blank" style={{ float: "right", fontSize: "smaller" }}>source</a>

### M5Evaluation

> ```text theme={null}
>  M5Evaluation ()
> ```

*Initialize self. See help(type(self)) for accurate signature.*

### URL-based evaluation

The method `evaluate` from the class
[`M5Evaluation`](https://Nixtla.github.io/datasetsforecast/m5.html#m5evaluation)
can receive a url of a [submission to the M5
competiton](https://github.com/Nixtla/m5-forecasts/tree/main/forecasts).

The results compared to the on-the-fly evaluation were obtained from the
[official
evaluation](https://github.com/Mcompetitions/M5-methods/blob/master/Scores%20and%20Ranks.xlsx).

```python theme={null}
m5_winner_url = 'https://github.com/Nixtla/m5-forecasts/raw/main/forecasts/0001 YJ_STU.zip'
winner_evaluation = M5Evaluation.evaluate('data', m5_winner_url)
# Test of the same evaluation as the original one
test_close(winner_evaluation.loc['Total'].item(), 0.520, eps=1e-3)
winner_evaluation
```

### Pandas-based evaluation

Also the method `evaluate` can recevie a pandas DataFrame of forecasts.

```python theme={null}
m5_second_place_url = 'https://github.com/Nixtla/m5-forecasts/raw/main/forecasts/0002 Matthias.zip'
m5_second_place_forecasts = M5Evaluation.load_benchmark('data', m5_second_place_url)
second_place_evaluation = M5Evaluation.evaluate('data', m5_second_place_forecasts)
# Test of the same evaluation as the original one
test_close(second_place_evaluation.loc['Total'].item(), 0.528, eps=1e-3)
second_place_evaluation
```

By default you can load the winner benchmark using the following.

```python theme={null}
winner_benchmark = M5Evaluation.load_benchmark('data')
winner_evaluation = M5Evaluation.evaluate('data', winner_benchmark)
# Test of the same evaluation as the original one
test_close(winner_evaluation.loc['Total'].item(), 0.520, eps=1e-3)
winner_evaluation
```

### Validation evaluation

You can also evaluate the official validation set.

```python theme={null}
winner_benchmark_val = M5Evaluation.load_benchmark('data', validation=True)
winner_evaluation_val = M5Evaluation.evaluate('data', winner_benchmark_val, validation=True)
winner_evaluation_val
```

## Kaggle-Competition-M5 References

The evaluation metric of the Favorita Kaggle competition was the
normalized weighted root mean squared logarithmic error (NWRMSLE).
Perishable items have a score weight of 1.25; otherwise, the weight is
1.0.

$ NWRMSLE = \sqrt{\frac{\sum^{n}_{i=1} w_{i}\left(log(\hat{y}_{i}+1)  - log(y_{i}+1)\right)^{2}}{\sum^{n}_{i=1} w_{i}}}$

|                                Kaggle Competition Forecasting Methods                                | 16D ahead NWRMSLE |
| :--------------------------------------------------------------------------------------------------: | :---------------: |
| [LGBM](https://www.kaggle.com/shixw125/1st-place-lgb-model-public-0-506-private-0-511/comments) \[1] |       0.5091      |
|                       [Seq2Seq WaveNet](https://arxiv.org/abs/1803.04037) \[2]                       |       0.5129      |

1. [Corporación Favorita. Corporación favorita grocery sales
   forecasting. Kaggle Competition Leaderboard,
   2018.](https://www.kaggle.com/c/favorita-grocery-sales-forecasting/leaderboard)
2. [Glib Kechyn, Lucius Yu, Yangguang Zang, and Svyatoslav Kechyn.
   Sales forecasting using wavenet within the framework of the Favorita
   Kaggle competition. Computing Research Repository, abs/1803.04037,
   2018](https://arxiv.org/abs/1803.04037).