FreeSolv SMILES Hydration Free Energy Baseline

Build a reproducible baseline model for the FreeSolv hydration free energy dataset. The model must predict experimental hydration free energy from SMILES strings, report RMSE on a fixed held-out test split, and include enough artifacts for the result to be checked. Among valid Submissions, the lowest test RMSE wins.

The task is a small molecular property prediction benchmark. Solvers must train a model that takes the smiles column as input and predicts the expt column from the fixed FreeSolv CSV snapshot listed below. The expt values are experimental hydration free energies in kcal/mol.

The held-out test split is deterministic and fixed by this bounty: using the CSV row order after parsing the header, rows with zero-based row index i where i % 5 == 0 are the test set; all other rows are the development set. The reported RMSE must be computed only on that test set.

Solvers may choose any modeling method that can be rerun from their submitted files, including descriptor-based baselines, fingerprints, graph models, or simpler statistical models. A valid Submission must not train, tune, select features, choose checkpoints, or otherwise optimize using the test-set expt values.

Each required deliverable must be included as a plain file in the Submission. Do not submit directories; use the exact filenames below.

The Submission may include additional plain files if needed, such as requirements.txt or helper source files. All required files must fit within Elgora's Submission limits.

The authoritative input is the byte sequence matching the specified SHA-256. Bytes with a different SHA-256 are not the fixed dataset for this bounty.

Solvers must describe how the test set was kept out of training and model selection. If the method uses randomness, the Solver must report the seed or explain why the result is deterministic. If external packages, pretrained models, molecular descriptors, or learned representations are used, the Solver must identify them and explain whether they were trained on FreeSolv test labels.

A file hash proves only byte identity. It does not prove that code was run, that test labels were not used, or that a modeling claim is scientifically valid. Those claims must be supported by the submitted code, outputs, and explanation.

The primary score is RMSE on the fixed held-out test split:

RMSE = sqrt(mean((predicted_hydration_free_energy - true_expt_hydration_free_energy)^2))

Use all test rows defined by i % 5 == 0. Lower RMSE is better. The score should be reported with at least six decimal places. The governing score for eligibility and ranking is the RMSE calculated from the submitted predictions.csv; if that value differs from the report, the value derived from predictions.csv governs.

A valid Submission with the lowest governing test RMSE wins. Let best_rmse be the lowest governing RMSE among all valid Submissions. Every valid Submission with governing RMSE no more than best_rmse + 0.000001 is in the tie group. If the tie group contains more than one Submission, the winner is the Submission in that group whose lowercase Solver address sorts first in ascending order. If only one Submission is valid, it wins. If no Submission is valid, the outcome is no_valid_submission.

A Submission is ineligible regardless of RMSE if a required deliverable is missing after the Submission is successfully retrieved and opened, if predictions.csv is missing, malformed, omits test rows, adds non-test rows, or lacks the values needed to calculate RMSE, if the code and outputs materially disagree, or if the submitted evidence shows that test-set target values were used for training, tuning, checkpoint selection, feature selection, or manual prediction adjustment.

A Submission is also ineligible if it relies on data unavailable from the fixed input and submitted files in a way that leaves its reported result unsupported by its own evidence, or if it attempts to replace the fixed dataset or split with another benchmark.

This bounty does not pay for new wet-lab measurements, new dataset curation, hidden-test benchmarking, leaderboard-only claims, clinical or therapeutic conclusions, or models that cannot be checked from submitted artifacts.

For each opened Submission, the result is defined against the fixed SAMPL.csv snapshot and the fixed test split defined in this page. Eligibility and ranking use the RMSE calculated from the submitted predictions.csv. The submitted code and provenance files must support reproduction of those predictions and the reported metric from the fixed input and submitted files.