Version	1.4
Created	2023/06/07 - Michael P. Dubner (https://github.com/pyhedgehog)
Modified	2024/03/05 - Michael P. Dubner (https://github.com/pyhedgehog)

Abstract

This SIP (Saltcorn Improvement Proposal) has goal to make packs more developer-friendly format.

Motivation

Packs pros

Features that should be keept:

1. Pack has simple representation that can be constructed manually/programmatically.
2. Pack has text representation that can be stored in VCS.
3. Snapshots are packs.
4. On pack re-import existing triggers are overrided.
5. Pack format is (theoretically) extensible.

Packs cons

Defects that should be fixed:

1. Packs can be imported only once (#1507)
   1. Existing table cause pack import to fail.
   2. Existing view/page silently ignored.
2. Packs export can't be easily repeated (#1505)
3. Packs are not self-consistent. There are configuration parts not included:
   1. Custom events (#1806)
   2. Menu items (#2090)
   3. Log settings
   4. Tags of objects in pack (done in #2053)
   5. Language translations (#267)
   6. Content of some tables (aka reference tables). I.e. Aspect table in Roadmap pack.
   7. Available NPM packages required in triggers.
4. Packs imported from cli (saltcorn install-pack -f ...) require saltcorn restart.

Extra ideas

1. Plugins or patches for packed application migration. (see #1475-comment)

Proposal

1. Update existing views, pages (and maybe library items) like triggers.
2. Add ability to define default value for required column in pack.
3. Update existing table definition if posiible fail otherwise.
4. Add "create pack" to tag interface. This will simplify re-export. (done in #2053)
5. Send reload signal from cli install-pack command to server (i.e. SIGHUP which should be handled beforehand).

Format changes

1. Add format version field ("format": 1) while absence of this field implies format=0, i.e. old format.
2. Add version field ("version": "1.0" by default).
3. Add custom_events field as list of {"name":"...","hasChannel":false}.
4. Add menu_items field.
5. Add localizer_strings field with translation strings of exported views and pages and menu items.
6. Add data object field with table_name:[rows] fields.

Import changes

1. Update (aka replace) existing views, pages (and maybe library items) like triggers.
2. Update existing table definition if posiible fail (and rollback) whole pack import otherwise.
3. Apply (add or update) configuration changes (custom_events, menu_items, localizer_strings).
4. Import data using upsert idiom.
5. Process whole import in single transaction to be able to rollback it.

Interface changes

1. Add ability to define default value for required column in pack (done in #1824 and #2117).
2. Change pack export interface as a whole:
   1. Add pack re-export. Let's add first step with selection "Export new pack" (name it) and "Re-export old pack" (select from installed ones).
   2. Add versioning. For new packs versions defaults to 1.0, for re-export we can ever add detection of major changes (fields added).
   3. Mark pack installed on export including version bump (update installed_packs config).
   4. Add "create pack" to tag interface. This will simplify re-export with version increment.
   5. Add "dump data" as an option to exported tables excluding readonly (external or provider) tables, tables without PK and tables with high rows count (i.e. >1M).
3. Change pack import interface:
   1. Add version to installed_packs config.
   2. Add checkbox "keep going" to revert to old import behaviour (without transaction).

Misc changes

1. Handle reload signal (i.e. SIGHUP). Windows is problem as usual.
2. Send reload signal from cli install-pack command to server (if pid os known).