There are various database frameworks, some popular ones being document-based (e.g., MongoDB), relational (e.g., postgreSQL), and knowledge graph-based (e.g., neo4j).
Generally a document-based one is easiest to get started with (especially given the parallels and user familiarity with Python dictionaries), but tends to adapt poorly as needs change and schema are updated.
SQL and other relational databases have a long history, and while the entry barrier is generally higher, these seem to do relatively well when adjusting to dynamically changing needs. Data (characterization results, compositions and processing conditions, and other data and metadata) for the physical sciences are inherently relational.
Graph-based databases often evoke somewhat nebulous terms to some such as ontologies and knowledge graphs. However, generally people who come to understand what these are tend to agree that it’s the “long-term dream”, in some sense taking relational a step further. However, these also seem to have one of the highest learning curves and a fair bit of ambiguity and careful thought required in terms of the design choices of the database (again, especially thinking about ontologies).
Depending on the discipline, the data types and data formats can vary a lot. It gets more complex when you start thinking about integrations with electronic lab notebooks or laboratory information management systems or direct upload of data by devices. How are you approaching this in your labs? What is working well, or not working so well? Even with highly custom implementations, it would help to be aware of what thinking went into the design decisions.
Cc @willigo09. Also, Sergio and Matthias Popp.