==========================
Solutions Backend Programs
==========================

Several backend programs are executed on a periodic schedule to maintain the
Solutions database.  This document explains how Solutions users with sufficient
permissions can execute specific backend tasks whenever this is required.

IMPORTANT: The backend programs will do what you ask for as fast as possible,
without any warnings about potential dangers.  It is recommended that you review
this document before using the programs.  In addition, before you hit the ENTER
key after typing a command, please read through what you have typed to check
that everything is spelled correctly and the whole line is as intended.


Setup
=====

The backend programs are installed on the Linux host
``math-apps1.sas.rutgers.edu``, so you need to login to this computer to use
them::

   ssh math-apps1.sas.rutgers.edu

The first time you login, you also need to setup some information required by
the programs.  This is done by entering these commands::

   PATH=$PATH:/home/msolutions/bin
   setup-secure

You can rerun the ``setup-secure`` command any time your information changes.


Backend programs
================

After you have executed ``setup-secure``, you will be able to use the commands
listed in this section.  Many of these commands can accept options that modify
their behavior.  In particular, many commands accept the option ``--help``,
which causes them to print the list of all options they accept and terminate
without doing anything else.  For example, the following will print the options
accepted by the ``rpscan`` command::

   rpscan --help

The commands that accept the ``--help`` option are marked with ``[--help]`` in
the list below.  If you use a command without specifying any options, then it
will be executed with the same parameters as when it is executed automatically
by the system.

List of commands
----------------

``status``
   Print a summary of the backend log files.

``sql1 [--help]``
   Login to the LIVE database server ``math-sql1.sas.rutgers.edu``.

``sql2 [--help]``
   Login to the TEST database server ``math-sql2.sas.rutgers.edu``.

``dslogin``
   Login to Rutgers Data Services.

``backup``
   Create a backup copy of all tables in the Solutions database.  The backup
   copy will be stored in the subdirectory ``backup`` of your home directory
   (that is, ``/home/netid/backup`` with your netid substituted).  This
   subdirectory will be created if it does not exist.

``old2new [--help]``
   Translate some tables from Access-Solutions to Python-Solutions version,
   for example :ref:`tbl1_People` to :ref:`people`.

``copy2sql [--help]``
   Copy the most important log files to the database table :ref:`logfiles`, so
   that they are visible in ``LogViewer``.

``getforms [--help]``
   Connect to Joomla Administrator to download the information collected by the
   Joomla forms.  (No longer used.)

``copyforms``
   Copy the information collected by Joomla forms from Joomla database tables
   provided by SAS-IT.

``rpscan [--help]``
   Import new student requests into the database, so that they are visible in
   ``RequestPortal``.  This program uses the information collected by
   ``copyforms``.

``getdataservices [--help]``
   Download information from Rutgers Data Services, including course schedules
   and information about buildings and rooms.

``mkschedule [--help]``
   Generate an HTML schedule of all mathematics courses.  The html files will be
   stored in the subdirectory ``schedule`` of your home directory.  This
   subdirectory will be created if it does not exist, and all files in this
   ``schedule`` subdirectory will be deleted each time mkschedule is executed.

``copyschedule``
   Copy the schedule files generated by ``mkschedule`` to iw3.math.rutgers.edu.
   They will then be visible at the URL https://iw3.math.rutgers.edu/schedule/.

``getcssinfo [--help]``
   Collect the information available from the course list on the CSS website and
   store it in the database table :ref:css_info`.  This table is used to show
   projected enrollments in ``RequestPortal``.

``getcss [--help]``
   Collect the information available when viewing or editing the individual
   courses on the CSS website and store it in the database table
   :ref:`css_sections`.

``putcss [--help]``
   Update instructors on the CSS website to match those in the database table
   :ref:`css_upload`.  No changes will be made if :ref:`css_upload` contains a
   copy of :ref:`css_sections` (assuming that ``getcss`` has just been run and
   no changes have been made on the CSS website).

``fixcss``
   Attempt to work around errors in Access-Solutions by changing fields
   in :ref:`css_upload`.  This includes removing instructors with netid
   ``TBD`` and trying to fix the mode codes.

``checkcss [--help]``
   Check if the data in :ref:`css_sections` satisfies a list of rules, warn
   about violations.  The warnings will show up in ``checkcss.log`` in
   ``LogViewer``.

``getfinals [--help]``
   Download the finals schedule and store it in the database table
   :ref:`exam_schedule`.

``getsummer [--help]``
   Download the summer session schedule and store it in the database table
   :ref:`summer_section_dates_inbound`, which is used by the Access
   ``ControlCenter``.


Finding information about the backend programs
==============================================

To figure out which programs are executed automatically, and at which times, you
can inspect the files in ``/home/msolutions/etc`` and its subdirectories.  The
relevant subdirectories have names starting with ``cron.``.  You can see the
list by typing::

   ls /home/msolutions/etc

Show which commands are executed every 15 minutes::

   ls /home/msolutions/etc/cron.15min

Show which commands are executed every day at around 1 AM::

   ls /home/msolutions/etc/cron.daily01

Show exactly when the programs in each directory are scheduled to run::

   cat /home/msolutions/etc/crontab

In the table printed by this command, the first column determines the minute and
the second column determines the hour.


Recipes
=======

This section describes some procedures that can be used to carry out specific
maintenance tasks.

Clear CSS instructors
---------------------

To clear all instructors in CSS for subject code 01:640 in Fall 2020, enter the
following commands in order.  You can optionally execute ``getcss`` first if you
suspect that the table :ref:`css_sections` is out of date.  Please avoid using
this procedure around 5 AM in the morning, since at that time it may conflict
with the automatic CSS sync::

   sql1 -D mathdb -e "delete from css_upload"
   sql1 -D mathdb -e "insert into css_upload select * from css_sections"
   sql1 -D mathdb -e "update css_upload set netid1=NULL, netid2=NULL"
   putcss 9-2020:01:640 --dry-run
   putcss 9-2020:01:640
   sql1 -D mathdb -e "delete from css_upload"

Note: The option ``--dry-run`` will cause ``putcss`` to show the intended
changes to CSS without carrying them out.  This is useful for making sure that
you agree with the changes before the second ``putcss`` command is executed.
The dry-run command can be skipped if you are confident that everything is fine.

Note: Do not forget to enter the semester-subject argument (``9-2020:01:640``).
If no such argument is given, then ``putcss`` will operate on all semesters that
Solutions is currently working on and clear the instructors in all these
semesters.

If you only want to clear the instructors of some courses, then use the
``--courses`` option to ``putcss``.  For example::

   putcss 9-2020:01:640 --courses 250,300-311 --dry-run
   putcss 9-2020:01:640 --courses 250,300-311