Welcome to REMI! This site will host a diverse collection of scripting notebooks (Jupyter, Matlab LiveScripts, etc.) for collecting, pre-processing, analyzing, and visualizing materials data. Notebooks are curated using tags aligned to Materials Science and Data Science topics. If you know of notebooks that would be great additions to REMI, please click here. We are also working to integrate a communication platform for holding discussions.
Above you will find links for learning resources, methods to contribute to REMI, upcoming workshops in pertinent fields, and open positions in academia and national labs.
Many scientists would like to apply machine learning to their research, but they don’t know how to start. Typically, we start learning something new by picking up a learning resource (e.g. a textbook or intro paper) and working through some examples. If the examples are particularly useful, we build off of them to solve our own challenges.
REMI emerged from the realization that both experts and novices wanted examples of using machine learning for science. Meanwhile, lots of experts are developing digital notebooks (e.g. Jupyter) to demonstrate step-by-step data collection, pre-processing, analysis and visualization. However, before REMI there were no indexed repositories of notebooks with such examples and no community to maintain, build or learn from these resources. We hope that REMI can fill this gap, enabling scientists to more easily pick up machine learning while also helping to build a community for sharing knowledge, discussing, debating, establish collaborations, and benchmarking methods.
| Resource Name | Type | Collection | Data Science Tags | Material Science Tags |
|---|---|---|---|---|
| Automatminer Basic Tutorial | Tutorial | Matminer | Platform:MatBench DimensionReduction:FeatureReducer Regression:RandomForest Preprocessing:Featurizer TargetProperty:Bandgap | |
| Data Retrieval Basics | Tutorial | Matminer | Platform:MaterialsProject Platform:Citrination Platform:MaterialsPlatformForDataScience Platform:Materials Data Facility | Element:Pb Element:Te |
| Plot and compare experimental and computational bandgaps | Tutorial | Matminer | Platform:MaterialsProject Platform:Citrination | Computation:DFT Property:bandgap |
| Interacting with Jarvis via MDF Search | Tutorial | Matminer | Platform:JARVIS Platform:MaterialsDataFacility | Computation:DFT Property:bandgap Property:ShearModulus Property:BulkModulus |
| Uranium-oxygen bond length analysis using MPDS | Tutorial | Matminer | Platform:MaterialsPlatformForDataScience | Element:Ur Element:O Property:BondLength |
| Visualization using FigRecipes | Tutorial | Matminer | ||
| Advanced Visualization using FigRecipes | Tutorial | Matminer | MaterialClass:Thermoelectric | |
| Matminer introduction - Predicting bulk modulus | Tutorial | Matminer | Regression:LinearRegression Regression:RandomForest Preprocessing:Featurizer | MaterialClass:InorganicCrystallineCompounds Property:BulkModulus |
| Train a Model to Predict Formation Energy using the OQMD | Example | Matminer | Platform:MaterialsDataFacility Platform:OpenQuantumMaterialsDatabase Regression:RandomForest Preprocessing:Featurizer | Property:FormationEnthalpy |
| Recreating Ling IMMI (2017) - Random Forest with Uncertainty | Example | Matminer | Platform:Citrination Regression:RandomForest Preprocessing:Featurizer ActiveLearning:MaximumExpectedImprovement ActiveLearning:MaximumLikelihoodOfImprovement | Property:ZT |
| Using sklearn Pipeline with matminer | Example | Matminer | Platform:Matminer Regression:LinearRegression Regression:RandomForest Preprocessing:Featurizer | |
| Using the Crystal Structure Representation of Ward et al. to predict formation enthalpy,using the FLLA dataset. | Example | Matminer | Regression:RandomForest Preprocessing:Featurizer | Property:FormationEnthalpy |
| Analyzing data in the JARVIS DFT dataset | Example | Jarvis | Platform:JARVIS | Computation:DFT Property:Bandgap |
| BoltzTrap Example | Example | Jarvis | Platform:JARVIS | Computation:DFT Property:Seebeck |
| JARVIS DFT Formation Energies Accuracy Check | Example | Jarvis | Platform:JARVIS | Computation:DFT Property:FormationEnergy |
| GPU can accelerate model training with respect to CPU using JARVIS-ML CFID dataset | Example | Jarvis | Platform:JARVIS Regression:GradientBoosting | Computation:DFT Property:FormationEnergy |
| JARVIS ML TensorFlow Example | Example | Jarvis | Platform:JARVIS Regression:DenseNet Preprocessing:StandardScaler | Computation:DFT Property:FormationEnergy |
| JARVIS ML Training GPU | Example | Jarvis | Platform:JARVIS Regression:GradientBoosting | Computation:DFT |
| JARVIS Wannier 90 Example | Example | Jarvis | Platform:JARVIS | Computation:DFT Property:BandStructure Property:DensityOfStates |
| Si band structure & density of states | Example | Jarvis | Platform:JARVIS | Computation:DFT Property:BandStructure Property:DensityOfStates |
| Simple Machine learning training example with CFID descriptors | Example | Jarvis | Platform:JARVIS Regression:GradientBoosting | Computation:DFT Property:FormationEnergy |
| Simple Silicon atomic structure and analysis example | Example | Jarvis | Platform:JARVIS | Element:Si Computation:DFT |
| Python for novice | Example | Jarvis | ||
| Introduction | Example | MDF Forge | Platform:MaterialsDataFacility | |
| Core Query Builder Functions | Example | MDF Forge | Platform:MaterialsDataFacility Platform:OpenQuantumMaterialsDatabase | Computation:DFT |
| General Helper Functions | Example | MDF Forge | Platform:MaterialsDataFacility Platform:OpenQuantumMaterialsDatabase Platform:MaterialsDataRegistry | Computation:DFT |
| Field Specific Helper Functions | Example | MDF Forge | Platform:MaterialsDataFacility Platform:OpenQuantumMaterialsDatabase | Computation:DFT |
| Data Retrieval Functions | Example | MDF Forge | Platform:MaterialsDataFacility | |
| Band Excitation data processing | Example | pycroscopy | Measurement:DCvoltageSpectroscopy Property:PFMHysteresis | |
| Band Excitation & Contact Kelvin Probe Force Microscopy (cKPFM) data processing | Example | pycroscopy | Preprocessing:PrincipalComponentAnalysisSmoothing DimensionReduction:PrincipalComponentAnalysis | Measurement:ContactKelvinProbeForceMicroscopy Measurement:BandExcitation Measurement:JunctionContactPotentialDifference |
| Band Excitation Relaxation Spectroscopy Data Processing | Example | pycroscopy | Measurement:PFM Property:BandExcitationRelaxationSpectra | |
| G-Mode KPFM with Fast Free Force Recovery (F3R) | Example | pycroscopy | Preprocessing:PrincipalComponentAnalysisSmoothing DimensionReduction:PrincipalComponentAnalysis | Measurement:GKPFM |
| Static force spectroscopy simulation over a viscoelastic material | Example | pycroscopy | Computation:StaticForceSpectroscopy MaterialClass:Viscoelastic | |
| Introduction to dynamic AFM simulations | Example | pycroscopy | Computation:DynamicAFM | |
| Dynamic atomic force microscopy simulations over a viscoelastic material | Example | pycroscopy | Computation:DynamicAFM MaterialClass:Viscoelastic | |
| AFLOW.org database and APIs | Example | AFLOW | Platform:AFLOW | |
| Structural analysis and prototyping | Example | AFLOW | Platform:AFLOW | Property:Structure |
| AFLOW machine learning | Example | AFLOW | Platform:AFLOW Regression:GradientBoosting Regression:PropertyLabeledMaterialsFragments Preprocessing:PropertyLabeledMaterialsFragments | Property:Electronic Property:ThermoMechanical |
| Density of States | Example | Plotly | Platform:MaterialsProject | Computation:DFT Property:DensityOfStates Property:BandDiagram |
| EELS analysis | Example | hyperspy | FileFormat:HDF5 | MaterialClass:PerovskiteOxides Measurement:EELS |
| SEM EDS 4D visualization | Example | hyperspy | FileFormat:RPL Preprocessing:PrincipalComponentAnalysisSmoothing DimensionReduction:PrincipalComponentAnalysis | Element:Ni MaterialClass:Superalloy Measurement:4D-EDS-SEM |
| TEM EDS nanoparticles | Example | hyperspy | FileFormat:HDF5 Preprocessing:WatershedTransformation DimensionReduction:IndependentComponentAnalysis | Element:Fe Element:Pt MaterialClass:CoreShellNanoparticles Measurement:EDS-TEM |
| Holography | Example | hyperspy | FileFormat:HDF5 | Element:Fe MaterialClass:Needle Measurement:OffAxisElectronHolography |
| Getting Started | Tutorial | hyperspy | FileFormat:HDF5 | |
| SVD and BSS | Example | hyperspy | FileFormat:HDF5 Preprocessing:PrincipalComponentAnalysisSmoothing DimensionReduction:PrincipalComponentAnalysis DimensionReduction:IndependentComponentAnalysis DimensionReduction:NonNegativeMatrixFactorization | Measurement:EELS |
| Fitting tutorial | Tutorial | hyperspy | FileFormat:HSPY Preprocessing:FeatureExtraction | |
| Online Robust PCA | Example | hyperspy | FileFormat:HDF5 Preprocessing:PrincipalComponentAnalysisSmoothing DimensionReduction:PrincipalComponentAnalysis DimensionReduction:IndependentComponentAnalysis DimensionReduction:OnlineRobustPrincipalComponentAnalysis | |
| Working with image data | Example | hyperspy | FileFormat:HSPY | Element:Sr Element:Ti Element:O Measurement:HighResolutionScanningTEM |
| Working with image data | Example | hyperspy | ||
| How to extract or plot the NiO band structure from a VASP calculation using pymagen | Example | matgenb | Platform:MaterialsProject | Element:Ni Element:O Computation:DFT Property:BandStructure Property:DensityOfStates |
| core functionality of pymatgen,including the core objects representing Elements,Species,Lattices,and Structures | Example | matgenb | FileFormat:POSCAR FileFormat:CIF | |
| calculate reaction energies using the Materials API and pymatgen | Example | matgenb | Platform:MaterialsProject | Element:Ca Element:O Property:ReactionEnergy |
| how to plot an XRD plot for the two polymorphs of CsCl ($Pm\overline{3}m$ and $Fm\overline{3}m$) | Example | matgenb | Element:Cs Element:Cl Computation:XRD | |
| how to obtain an explaination of the different corrections being applied in the Materials Project | Example | matgenb | Platform:MaterialsProject | |
| Getting crystal structures from online sources | Example | matgenb | FileFormat:COD FileFormat:CIF | Element:Li Element:O |
| matgen's functionality in terms of creating and editing molecules,as well as its integration with OpenBabel | Example | matgenb | FileFormat:XYZ FileFormat:G09 FileFormat:PDB | Element:C Element:H |
| ordering of a disordered structure using pymatgen | Example | matgenb | FileFormat:CIF | |
| Plotting and Analyzing a Phase Diagram using the Materials API | Example | matgenb | Platform:MaterialsProject | Element:Ca Element:C Element:O Property:PhaseStability |
| Plotting the electronic structure of Fe | Example | matgenb | Property:ElectronicStructure | |
| Units | Example | matgenb | ||
| Data-driven First Principles Methods for the Study and Design of Alkali Superionic Conductors Part 1 - Structure Generation | Example | matgenb | FileFormat:CIF | Computation:DFT |
| Data-driven First Principles Methods for the Study and Design of Alkali Superionic Conductors Part 2 - Phase and Electrochemical Stability | Example | matgenb | Platform:MaterialsProject FileFormat:GZ | Element:Li Element:P Element:S Element:Cl Property:ElectrochemicalStability |
| Data-driven First Principles Methods for the Study and Design of Alkali Superionic Conductors Part 3 - Diffusivity and Ionic Conductivity | Example | matgenb | FileFormat:XML | Property:ActivationEnergy Property:IonicConductivity |
| how to use the output from VASP DFPT calculation and the phonopy interface to plot the phonon bandstructure and density of states | Example | matgenb | Computation:DFT Property:DensityOfStates Property:PhononBandStructure | |
| how you can obtain various data from the Materials Project using pymatgen's interface to the Materials API | Example | matgenb | Platform:MaterialsProject | Property:PhononBandStructure Property:ElasticConstants |
| Slab generation and Wulff shape | Example | matgenb | Element:Gd Element:O | |
| Inputs and Analysis of VASP runs | Example | matgenb | Element:Ni Element:Si Element:Li Element:Fe Element:P Element:O | |
| Running Jupyter Notebook on clusters | Example | matgenb | ||
| Analyze and plot band structures | Example | matgenb | Computation:DFT | |
| Plotting a Pourbaix Diagram | Example | matgenb | Platform:MaterialsProject | Element:Cu Element:O Computation:DFT Property:PourbaixDiagram |
| ChemEnv - How to automatically identify coordination environments in a structure | Example | matgenb | Platform:MaterialsProject FileFormat:CIF | Element:Si Element:O Computation:DFT |
| Computing the Reaction Diagram between Two Compounds | Example | matgenb | Platform:MaterialsProject | Element:V Element:P Element:C Element:H Element:O Computation:DFT |
| Plotting COHP from LOBSTER | Example | matgenb | Element:Fe Computation:DFT Property:CrystalOrbitalHamiltonPopulation | |
| Adsorption on solid surfaces | Example | matgenb | Platform:MaterialsProject | Computation:DFT Property:Adsorption |
| Structure Prediction using Pymatgen and the Materials API | Example | matgenb | Element:Ni Element:C Element:H Element:O | |
| Dopant suggestions using Pymatgen | Example | matgenb | Platform:MaterialsProject | Element:Sn Element:O Computation:DFT Property:NDopant Property:PDopant |
| How to use Boltztra2 interface | Example | matgenb | Platform:MaterialsProject | Computation:DFT Property:DensityOfStates Property:Transport Property:Seebeck |
| How to plot and evaluate output files from Lobster | Example | matgenb | FileFormat:LOBSTER | Element:Ga Element:As Computation:DFT Property:CrystalOrbitalHamiltonPopulation Property:DensityOfStates |
| Interface Reactions | Example | matgenb | Platform:MaterialsProject | Element:Li Element:Co Element:O Element:P Element:S Computation:DFT Property:InterfaceReactions |
| How to plot a Fermi surface | Example | matgenb | FileFormat:GZ FileFormat:CUBE | Element:Pb Element:Te Computation:DFT Property:FermiSurface |
| How to plot a Fermi surface | Example | matgenb | ||
| Experimental design with Citrination | Example | Citrine | Platform:Citrination ActiveLearning:MaximumLikelihoodOfImprovement | MaterialClass:Thermoelectric Property:ZT |
| Import Instron | Example | Citrine | Platform:Citrination FileFormat:PIF FileFormat:Instron | |
| Citrination t-SNE API | Example | Citrine | Platform:Citrination DimensionReduction:TDistributedStochasticNeighborEmbedding | Property:Bandgap |
| Citrination t-SNE API | Example | Citrine | ||
| Importing data from VASP calculations into Citrination | Example | Citrine | FileFormat:PIF Platform:Citrination | Computation:DFT |
| Working with PIFs | Example | Citrine | FileFormat:PIF Platform:Citrination | Element:Al Element:Cu Property:PhaseStability |
| Introduction to Queries | Example | Citrine | FileFormat:PIF Platform:Citrination | Element:Al Element:Cu Property:PhaseStability |
| Machine Learning on Citrination | Example | Citrine | FileFormat:PIF Platform:Citrination Platform:MaterialsProject Preprocessing:Featurizer | Element:Al Element:Cu Property:DensityOfStablePhases |
| Advanced PIF Tutorial | Example | Citrine | FileFormat:PIF Platform:Citrination | |
| Advanced Queries | Example | Citrine | FileFormat:PIF Platform:Citrination | MaterialClass:Oxides |
| Advanced Queries | Example | Citrine | ||
| Citrination Demo on Shape Memory Alloys | Example | Citrine | Platform:Citrination DimensionReduction:TDistributedStochasticNeighborEmbedding | MaterialClass:ShapeMemoryAlloys Property:TransitionTemperature |
| Citrination Demo on Shape Memory Alloys | Example | Citrine | ||
| Compare Band Gaps From Citrination and Materials Project | Example | Citrine | Platform:Citrination Platform:MaterialsProject | Property:Bandgap |
| Citrine Data Retrieval Example | Example | Citrine | Platform:Citrination | |
| Synthetic Data Demo | Example | Citrine | Platform:Citrination DimensionReduction:TDistributedStochasticNeighborEmbedding | |
| Synthetic Data Demo | Example | Citrine | ||
| NANO106 - Symmetry Computations on m3¯¯¯m (Oh) Point Group | Example | MaterialsVirtualLab | SymmetryClass:m-3mPointGroup Property:Symmetry | |
| NANO106 - Symmetry Computations on mmm(D2h) Point Group | Example | MaterialsVirtualLab | SymmetryClass:mmmPointGroup Property:Symmetry | |
| NANO106 - Symmetry Computations on mmm(D2h) Point Group | Example | MaterialsVirtualLab | ||
| Using pymatgen to mine the Materials Project | Example | MaterialsVirtualLab | Platform:MaterialsProject | MaterialClass:Inorganic |
| plot representation quadrics for some rank 2 tensors | Example | MaterialsVirtualLab | ||
| plot an XRD plot for the two polymorphs of CsCl (Pm3¯¯¯m and Fm3¯¯¯m) by computing the structure factors | Example | MaterialsVirtualLab | Element:Cs Element:Cl Computation:XRD Property:StructureFactor | |
| plot an XRD plot for the two polymorphs of CsCl (Pm3¯¯¯m and Fm3¯¯¯m) by computing the structure factors | Example | MaterialsVirtualLab | ||
| Bayesian Optimization and Regression Analysis for Ferromagnetic Materials | Example | Regression:GaussianProcess ActiveLearning:MaximumExpectedImprovement | Element:Fe Element:Ga Element:C MaterialClass:Magnetostriction Measurement:MagnetostrictionCantileverDisplacement Property:Magnetostriction |