Content packs

Why content packs?

For 0.15 and below, we started packaging assessment exercise resources such as PNGs and images into their own zip file (called the assessment zip). These need to be installed before KA Lite is usable. We also had language packs -- files that are useful for different languages, such as subtitles, translation catalog files, and dubbed video mappings.

For 0.16, we decided to combine both language packs and the assessment zip into one pack, called the content pack. The reasons are:

• performance: previously, since translation and the topic tree bundling logic was separate, translation of the topic tree, as well as topic availability, has to be done on the fly. Now with the topic tree pre-translated, stored into database, and availability marked to an extent, the KA Lite server has to do less work, especially during startup.
• non-english installer bundles: with translation files bundled in with assessment resources, it will now be easier to create KA Lite installers with different languages bundled in.

Terminology

• assessment resource zip: the zip file used for bundling assessment resources. Only contains assessment resources, and assessmentitems.json, which contains individual questions for each exercise. This is the only zip file needed by KA Lite versions 0.15 and below to work.

• language packs: zip files containing language related resources. These include:

  • catalog files, for translating the interface and topic tree.
  • video subtitles
  • dubbed video mapping, to allow the admin to download videos dubbed in the language currently activated.

• topic tree JSON files: for 0.15 and below, the Khan Academy topic tree data has been stored in 3 different JSON files: exercise.json, topics.json, and content.json. These are read and translated during startup for all languages installed.

• unpack_assessment_zip: the management command used to extract the contents of the assessment resource zip, and place it in the right location to make it serveable by KA Lite.

• languagepackdownload: the management command used to extract the contents of a language pack, and place it in the right location. This makes them usable by Django and servable by KA Lite.

• content pack: A zip file that contains the data from the assessment resource zip, language packs, and the topic tree JSON files. The most important thing it contains is the content db, which is the topic tree in a SQLite database, translated and availability annotated. The content db also has the assessment item metadata, also translated. For non-english content, each video node has also been mapped to its dubbed counterpart.

A content pack may also have the following contents:

• catalog files, for on-the-fly interface translation
• video subtitles
• assessment resources

0.16

Lay of the land

As a transitionary release, 0.16 will have both unpack_assessment_zip and retrievecontentpack, to ensure that we have enough time for testing the installers. We have the following content packs available:

• en.zip: the full content pack. Includes the content db, and assessment resources. About 500 MB in size.
• en-minimal.zip: the minimal version of en.zip. Only includes the content db. This can be downloaded by adding the --minimal flag to retrievecontentpack, like so:

bin/kalite manage retrievecontentpack download en --minimal

• {fr,zh,pt-BR,de,etc.}.zip: the various content packs, meant to replace 0.15's language packs. Includes the content db, interface catalog files, and subtitles.

Migrating from unpack_assessment_zip to retrievecontentpack

As of this writing, the output of make assets is an sdist prebundled with the content db. To achieve that, it runs retrievecontentpack download en --minimal, fetching en-minimal.zip and then packaging the content db with the final sdist.

For installers, there are two options you can do, if you're using the sdist: continue using unpack_assessment.zip (link here), or bundle in en.zip (link here) and unpack it using kalite manage retrievecontentpack local en <en.zip path>. Both have the same end result of including both content.db and the assessment resources. If you're not using the sdist, you can run just the retrievecontentpack step, or find a way to bundle both the content db, and then run use unpack_assessment_zip to unpack the assessment resource zips separately.

For the developer, it's recommended you use bin/kalite manage retrievecontentpack download moving forward.