Update README.md

README.md

@@ -1,3 +1,144 @@
----
-license: mit
----
+---
+license: mit
+base_model: ESFM/ESFM_s_nm_pre
+tags:
+- weather-forecasting
+- climate
+- earth-system
+- era5
+- pytorch
+---
+
+# Model Card for ESFM/ESFM_s_nm
+
+Final released deterministic ESFM small checkpoint trained without the missing-data masking protocol. It is the dense-input ERA5 baseline and forecasts global atmospheric fields six hours ahead.
+
+> **Checkpoint selection:** Six-hour deterministic forecasting from complete ERA5-style global inputs. This is also the deterministic base for the eight-member `ESFM_s_nm_ens` checkpoint.
+
+## Model Details
+
+- **Developed by:** The ESFM research team, with the full contributor and author lists linked below.
+- **Shared by:** [ESFM on Hugging Face](https://huggingface.co/ESFM)
+- **Model type:** Final deterministic dense-input ERA5 checkpoint (ESFM_s,kd*); modified 3D Swin-UNet encoder-decoder
+- **Model size:** Approximately 115 million parameters
+- **Masking protocol:** No observation masking
+- **Forecast lead time:** 6 hours
+- **License:** MIT
+- **Repository:** https://huggingface.co/ESFM/ESFM_s_nm
+
+### Model Sources
+
+- **Code:** https://github.com/swiss-ai/ESFM
+- **Paper:** https://arxiv.org/abs/2605.00850
+- **Project page:** https://swiss-ai.github.io/ESFM/
+
+The paper is currently available as an arXiv preprint.
+
+## Uses
+
+### Direct Use
+
+Six-hour deterministic forecasting from complete ERA5-style global inputs. This is also the deterministic base for the eight-member `ESFM_s_nm_ens` checkpoint.
+
+### Downstream Use
+
+Finetuning or rollout research where complete dense initial conditions are available.
+
+### Out-of-Scope Use
+
+Not intended for inputs with missing variables, pressure levels, or spatial regions, nor for sparse station or satellite inputs. Use the matching specialized or masked checkpoint instead.
+
+## Bias, Risks, and Limitations
+
+The model inherits ERA5 biases, trails larger specialist systems on clean dense benchmarks, and was not primarily optimized for long autoregressive rollouts. It is a research model, not an operational warning system.
+
+All ESFM checkpoints are research artifacts. Users should validate forecasts for their variables, regions, seasons, lead times, missingness pattern, and decision context. Do not use the model as the sole basis for safety-critical decisions.
+
+## How to Get Started
+
+The checkpoint is not packaged as a Hugging Face Transformers from_pretrained model. Construct the ESFM architecture with the matching repository config, then load the state dictionary. The released notebook contains the complete download, model-construction, normalization, and inference workflow.
+
+```bash
+git clone https://github.com/swiss-ai/ESFM.git
+cd ESFM
+# Open notebooks/inference_ESFMs_on_ERA5.ipynb
+```
+
+In the notebook, set:
+
+```python
+EXPERIMENT_NAME = "ESFM_s_nm"
+```
+
+To download the weights directly:
+
+```python
+from huggingface_hub import hf_hub_download
+
+model_name = "ESFM_s_nm"
+weights_path = hf_hub_download(
+    repo_id=f"ESFM/{model_name}",
+    filename=f"{model_name}.safetensors",
+)
+print(weights_path)
+```
+
+Set `EXPERIMENT_NAME = "ESFM_s_nm"` in `notebooks/inference_ESFMs_on_ERA5.ipynb`, or run `python inference.py --config configs/config_ESFM_s_nm.yaml` after configuring dataset paths.
+
+## Training Details
+
+### Training Data
+
+WeatherBench2 ERA5 at 0.25-degree resolution, trained on 1979 through 2020 and evaluated in the manuscript on held-out windows in 2023 and 2024.
+
+Dataset preprocessing and the exact variable registry are documented in the [ESFM repository](https://github.com/swiss-ai/ESFM) and [preprint](https://arxiv.org/abs/2605.00850).
+
+### Training Procedure
+
+Initialized through 40,000-step encoder knowledge distillation and 100,000-step ERA5 training without masking as `ESFM_s_nm_pre`, then continued for 10,000 steps as `ESFM_s_nm`. This experiment was run on 16 GPUs.
+
+- **Training objective:** Six-hour forecast learning, as specified above
+- **Nominal architecture:** ESFM small, approximately 115M parameters
+- **Software environment:** PyTorch/Lightning in the released NVIDIA PhysicsNeMo 25.03 container; lightning==2.5.1 is pinned in the Dockerfile
+- **Training regime:** Lightning `precision="32-true"` with FP32 parameters and optimizer state; selected model forward operations use CUDA BF16 autocasting through `torch.autocast(dtype=torch.bfloat16)`.
+
+## Evaluation
+
+The manuscript evaluates six-hour forecasts with latitude-weighted MAE and Pearson correlation and uses this checkpoint family as the dense-input comparison for masked ESFM. Detailed and evolving results remain in the linked preprint.
+
+The manuscript uses held-out temporal data and reports task-appropriate metrics: latitude-weighted MAE and Pearson correlation for gridded deterministic forecasts, relative MAE for MODIS comparisons, station metrics for station models, and CRPS for ensembles. Detailed values are intentionally not copied into this card.
+
+## Technical Specifications
+
+ESFM retains Aurora's 3D Swin-UNet backbone and adds variable-specific tokenization, axial attention across variables, perceiver aggregation across variables and pressure levels, learnable NaN tokens for missing patches, resolution-specific tokenizers where configured, and a decoder queried at target pressure levels. The small configuration uses a 256-dimensional embedding and approximately 115M parameters.
+
+## Environmental Impact
+
+- **Hardware type:** NVIDIA GH200 systems with four GPUs per node. This experiment was run on four nodes, totaling 16 GPUs.
+- **Total training time:** 20 hours
+- **Compute location:** Training used CSCS Alps infrastructure.
+
+## Citation
+
+```bibtex
+@misc{ozdemir2026esfm,
+  title={Earth System Foundation Model (ESFM): A unified framework for heterogeneous data integration and forecasting},
+  author={Firat Ozdemir and Yun Cheng and Salman Mohebi and Fanny Lehmann and Simon Adamov and Zhenyi Zhang and Leonardo Trentini and Dana Grund and Oliver Fuhrer and Torsten Hoefler and Siddhartha Mishra and Sebastian Schemm and Benedikt Soja and Mathieu Salzmann},
+  year={2026},
+  eprint={2605.00850},
+  archivePrefix={arXiv},
+  primaryClass={physics.ao-ph},
+  url={https://arxiv.org/abs/2605.00850}
+}
+```
+
+## More Information
+
+- [ESFM code and configurations](https://github.com/swiss-ai/ESFM)
+- [Current ESFM preprint](https://arxiv.org/abs/2605.00850)
+- [Data preprocessing scripts](https://github.com/swiss-ai/SwissClim_data_processing_scripts)
+- [Project page](https://swiss-ai.github.io/ESFM/)
+
+## Model Card Contact
+
+Firat Ozdemir: `firat.ozdemir@sdsc.ethz.ch`