Publications

Stéphane d'Ascoli, Sören Becker, Alexander Mathis, Philippe Schwaller, Niki Kilbertus (2023). ODEFormer: Symbolic Regression of Dynamical Systems with Transformers.

ArXiv Code Demo Twitter

Samy Jelassi, Stéphane d'Ascoli, Carles Domingo-Enrich, Yuhuai Wu, Yuanzhi Li, François Charton (2023). Length generalization in arithmetic transformers. arXiv preprint arXiv:2306.15400.

ArXiv

Stéphane d'Ascoli, Samy Bengio, Josh Susskind, Emmanuel Abbé (2023). Boolformer: Symbolic Regression of Logic Functions with Transformers. arXiv preprint arXiv:2309.12207.

ArXiv Code Demo Twitter

Stéphane d'Ascoli, Maria Refinetti, Giulio Biroli (2022). Optimal learning rate schedules in high-dimensional non-convex optimization problems. arXiv preprint arXiv:2202.04509.

ArXiv Code Twitter

Pierre-Alexandre Kamienny, Stéphane d'Ascoli, Guillaume Lample, François Charton (2022). End-to-end symbolic regression with transformers. Advances in Neural Information Processing Systems.

ArXiv Demo Code Talk Twitter

Stéphane d’Ascoli, Pierre-Alexandre Kamienny, Guillaume Lample, Francois Charton (2022). Deep symbolic regression for recurrence prediction. International Conference on Machine Learning.

ArXiv Yannic Kilcher Demo Code Talk Twitter

Stéphane d'Ascoli, Levent Sagun, Giulio Biroli, Ari Morcos (2021). Transformed CNNs: recasting pre-trained convolutional layers with self-attention. arXiv preprint arXiv:2106.05795.

ArXiv

Stéphane d'Ascoli, Marylou Gabrié, Levent Sagun, Giulio Biroli (2021). On the interplay between data structure and loss function in classification problems. Advances in Neural Information Processing Systems.

ArXiv NeurIPS Code Talk

Stéphane D'Ascoli, Hugo Touvron, Matthew L Leavitt, Ari S Morcos, Giulio Biroli, Levent Sagun (2021). ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases. Internation Conference on Machine Learning.

ArXiv ICML Blog post Code Long talk Short talk Twitter

Maria Refinetti, Stéphane D'Ascoli, Ruben Ohana, Sebastian Goldt (2021). Align, then memorise: the dynamics of learning with feedback alignment. International Conference on Machine Learning.

Arxiv ICML J. Phys. A Code Talk

Stéphane d'Ascoli, Levent Sagun, Giulio Biroli (2020). Triple descent and the two kinds of overfitting: where and why do they appear?. Advances in Neural Information Processing Systems.

ArXiv NeurIPS J. Stat Code Talk

Mario Geiger, Arthur Jacot, Stefano Spigler, Franck Gabriel, Levent Sagun, Stéphane d’Ascoli, Giulio Biroli, Clément Hongler, Matthieu Wyart (2020). Scaling description of generalization with number of parameters in deep learning. Journal of Statistical Mechanics: Theory and Experiment.

ArXiv J. Stat. Mech

Stéphane d'Ascoli, Maria Refinetti, Giulio Biroli, Florent Krzakala (2020). Double Trouble in Double Descent: Bias and Variance (s) in the Lazy Regime. International Conference on Machine Learning.

ArXiv ICML Medium Code Talk

Stéphane D’ascoli, Alice Coucke, Francesco Caltagirone, Alexandre Caulier, Marc Lelarge (2020). Conditioned Text Generation with Transfer for Closed-Domain Dialogue Systems. International Conference on Statistical Language and Speech Processing.

ArXiv Springer Code

Mario Geiger, Stefano Spigler, Stéphane d'Ascoli, Levent Sagun, Marco Baity-Jesi, Giulio Biroli, Matthieu Wyart (2019). Jamming transition as a paradigm to understand the loss landscape of deep neural networks. Physical Review E.

Arxiv Phys. Rev. E Code

Stéphane d'Ascoli, Levent Sagun, Giulio Biroli, Joan Bruna (2019). Finding the Needle in the Haystack with Convolutions: on the benefits of architectural bias. Advances in Neural Information Processing Systems.

ArXiv NeurIPS Slides Code

S Spigler, M Geiger, S d’Ascoli, L Sagun, G Biroli, M Wyart (2019). A jamming transition from under-to over-parametrization affects generalization in deep learning. Journal of Physics A: Mathematical and Theoretical.

ArXiv J. Phys. A Code

Stéphane d’Ascoli, Scott C Noble, Dennis B Bowen, Manuela Campanelli, Julian H Krolik, Vassilios Mewes (2018). Electromagnetic Emission from Supermassive Binary Black Holes Approaching Merger. The Astrophysical Journal.

ArXiv Ap. J. NASA press release Video