Python Management Commands

src/management/commands

Contains Django management command scripts.

backup.py

• Backs up static files, MySQL, Apache, and local config/ files into backup/ folder.
• Uses subprocess.check_call() for mysqldump, tarfile for creating TGZ files.
• Due to lack of permissions of www-data user to /etc/apache2/, Apache settings are archived by a cron job under ubuntu.
• The cron job is triggered weekly. Summary is sent to admin after job finish either via email.

cleanup.py

• Finds all JOB_ID records in JobIDs table that are older than KEEP_JOB months specified in config/cron.conf.
• Removes related result files in data/ and deletes the MySQL entry.
• Iterates over each file in data/ and finds results that are not recorded in MySQL (orphans), and remove the files.

dist.py

• Retrieves all releases of primerize package, and writes versions and texts into cache/stat_dist.json.
• Downloads all release ZIP files using curl with daslab-admin GitHub account credentials in config/oauth.conf. Files are saved to dist/. Releases already downloaded are skipped.
• The latest master branch is downloaded as dist/Primerize-master.zip. The LICENSE.md inside it is extracted and placed as dist/Primerize-LICENSE.md, which is used by license() in src/views.py for /license/.
• This command is triggered by GitHub webhooks, which is configured at https://github.com/DasLab/Primerize/settings/hooks, to keep the local ZIP files up-to-date. Webhooks are handled by git_hook() in src/api.py.

gdrive.py

• Uploads most recent backup onto daslab Google Drive and cleans up obsolete backup files.
• Uses subprocess.check_call() for drive. Files are renamed with date as prefix.
• Old files are determined by their last modified date and removed (default 45 days).
• The cron job is triggered weekly. Summary is sent to admin after job finish either via email or Slack.

restore.py

• Restores static files, MySQL, Apache, and local config/ files from local backup/ files.
• Uses subprocess.check_call() for mysql, tarfile and shutil for unpacking TGZ files.

versions.py

• Collects system installed package versions into cache/stat_sys.json and cache/stat_ver.json.
• Uses subprocess.Popen() for bash commands. Some commands need to write to cache/temp.txt temporarily for output handling. Some values, e.g. mod_wsgi, are implemented for production only. Commands for memory readings are dependent on DEBUG as well.
• Renders cache/sys_ver.txt (only in production) based on information in cache/stat_ver.json. It is displayed in bash upon login.
• Calls get_backup_stat() from src/console.py too.