Relational databases, NoSQL, and downstream usage in AI models

[aqui_c]

I wonder what databases are you setting up for storing experiments, protocols, and results, to maximize the effectiveness when trying to run (perhaps speculating about the future) AI models on the generated data.

We are heavily reliant on relational databases (we mostly use PostgreSQL, and SQLite for simpler logging, etc.) and was wondering if anyone has explored NoSQL (like MongoDB) what’s the rationale, and what’s the outcome so far.

[sgbaird]

Planning to come back to this, but some related discussion at:

Database best practices for SDLs and database types (e.g., document, relational, knowledge graphs)

There are various database frameworks, some popular ones being document-based (e.g., MongoDB), relational (e.g., postgreSQL), and knowledge graph-based (e.g., neo4j). Generally a document-based one is easiest to get started with (especially given the parallels and user familiarity with Python dictionaries), but tends to adapt poorly as needs change and schema are updated. SQL and other relational databases have a long history, and while the entry barrier is generally higher, these seem to do r…