Parikshith
41139f2fd0
Add 13 secondary indexes across 10 tables in the SQLite schema and 5 missing indexes to the PostgreSQL schema. All list endpoints (listSites, listResources, listClients, listRoles, listTargets, listUserOrgs) filter and join on these FK columns. In the SQLite schema, no secondary indexes existed at all on these foreign keys, forcing full sequential scans on every paginated request and session lookup. SQLite changes (13 new indexes): - sites.orgId - resources.orgId - targets.resourceId, targets.siteId - newt.siteId - clients.orgId, clients.userId - labels.orgId - olms.userId - session.userId - userOrgs.userId, userOrgs.orgId - roles.orgId PG changes (5 new indexes, rest already present): - labels.orgId - session.userId - userOrgs.userId, userOrgs.orgId - roles.orgId - olms.userId
Database
Pangolin can use a Postgres or SQLite database to store its data.
Development
Postgres
To use Postgres, edit server/db/index.ts to export all from server/db/pg/index.ts:
export * from "./pg";
Make sure you have a valid config file with a connection string:
postgres:
connection_string: postgresql://postgres:postgres@localhost:5432
You can run an ephemeral Postgres database for local development using Docker:
docker run -d \
--name postgres \
--rm \
-p 5432:5432 \
-e POSTGRES_PASSWORD=postgres \
-v $(mktemp -d):/var/lib/postgresql/data \
postgres:17
Schema
server/db/pg/schema.ts and server/db/sqlite/schema.ts contain the database schema definitions. These need to be kept in sync with with each other.
Stick to common data types and avoid Postgres-specific features to ensure compatibility with SQLite.
SQLite
To use SQLite, edit server/db/index.ts to export all from server/db/sqlite/index.ts:
export * from "./sqlite";
No edits to the config are needed. If you keep the Postgres config, it will be ignored.
Generate and Push Migrations
Ensure drizzle-kit is installed.
Postgres
You must have a connection string in your config file, as shown above.
npm run db:generate
npm run db:push
SQLite
npm run db:generate
npm run db:push
Build Time
There is a dockerfile for each database type. The dockerfile swaps out the server/db/index.ts file to use the correct database type.