SOLUTIONS SCHEDULE AND SYNCHRONIZATION OVERVIEW AND WORKFLOW¶
This document gives a basic high-level overview of the workflow involved in the Solutions Schedule and Synchronization Procedures
The synchronization process compares and contrasts data in the Solutions database with data in the Rutgers Course Scheduling System (CSS) and makes appropriate updates and reports of changes.
These notes are written by Alyssa Siegel, Solutions creator & developer.
Contact: siegel@math.rutgers.edu
PHASE 1: STARTING A NEW SEMESTER IN SOLUTIONS¶
To bring a new semester into Solutions for first time, office staff does the following:
In Solutions Database (SolutionsDashboard program), from “Course Scheduling” window, user selects term and year and clicks “New Semester”
A prompt will appear asking user to choose between two different ways to bring a new semester into Solutions. Choice #2 (import/export Excel files from CSS) is included only for historical purposes, and is included only if we need to go back to this technique for some reason (we might consider obsoleting this choice altogether at some point)
User should select choice #1, which says: “Mark this semester as PENDING. The next time a synchronize is run with Control Center (or manual import done), this semester will be built from scratch.
What happens after step 2: The new semster is added as a record in the tbl1_semester_statuses table with a special value in the column named “the_status”. This value, -200, translates to a global constant: SEMESTER_STATUS_FLAGGED_FOR_IMPORT
PHASE 2: DAILY SYNCHRONIZATION OF SOLUTONS DATA WITH CSS DATA¶
There are several inter-dependent processes that take place to support the sync process.
(2a) Supporting Python Programs¶
There are 2 main Python programs that are involved in this synchronization process:
getcss.py and putcss.py.
getcss.py pulls data from the CSS system for all semesters that Solutions cares about. This is determined by the value of the “the_status” field for each semester in the tbl1_semester_statuses table. Values of 0 and -200 in this field (meaning SEMESTER_STATUS_IN_PROGRESS or SEMESTER_STATUS_FLAGGED_FOR_IMPORT, respecitvely) will be the semesters that will be processed by getcss.py.
This program will save data found in the CSS schedule in the Solutions MySQL table css_sections.
putcss.py will compare data in the MySQL css_upload table with data in the css_sections table and post any updates (for example assigned instructors) to CSS, logging any changes that are made. (see 2b below for more information on how Solutions fills css_upload table).
These programs are set up to run (on linux machine) via crontab configuration (getcss.py runs every 4 hours. putcss.py runs once a day in an early-morning hour).
See solutions_python repository for more information about all of this.
(2b) Solutions Control Center¶
Solutions Control Center program is set up to run its synchronize process every morning in the 4am hour.
Here is an overview of what this means:
The program assumes that a fresh download of data from CSS has happened (via getcss.py) within the last 4 hours. This data is sitting in the css_sections table.
Data in css_upload table is emptied and refilled with a copy of data currently in css_sections table.
Code in the Control Center program will kick of the sync process for each semester that it currently cares about (see above).
Sync process will compare all data currently in Solutions with data downloaded from CSS and look for all the different types of changes that might happen (new or removed sections in CSS, instructors assigned in Solutions, canceled sections, change in location of class, etc). The css_upload table is modified to reflect changes that should be carried over into CSS.
IMPORTANT NOTE: css_upload will only be modified WITH THE CHANGES THAT SHOULD BE PROPOGATED TO CSS In our typical configuration and office procedures this will only amount to updates of INSTRUCTORS.
Discussion of all office procedures is beyond the scope of this document.
But a quick summary is this:
When an instructor discrepancy for UNDERGRADUATE courses is found between Solutions and CSS, then SOLUTIONS is assumed to be the correct instructor, and CSS will be updated to have the same instructor as Solutions (and css_upload is updated accordingly).
When any other discrepance is detected (room changes, new sections, etc), then CSS is assumed to have the correct values, and Solutions will be automatically updated when the control center sync code runs.
NEW SEMESTERS¶
When the control center processes a semester that is “flagged for import”, here is what will happen:
The data for the semester will be added to tbl1_schedule_data table. The status in tbl1_semester_statuses table will be changed to show pending status.
css_upload will be updated so all instructors for undergraduate courses are blank, which will force a “clear” of all instructors in CSS for beginning of new semester. When putcss.py is run, this should clear all instructors for the semester in CSS (a procedure office staff used to do manually).
PHASE 3: STAFF PROCEDURES¶
Here is a sample of how staff might work with control center:
When coming into office the morning, look the list of semesters that had syncs performed that morning
Look at the number of changes detected, and print or preview sync report to see what changes had happened
This report shows actions that were automatically taken, as well as actions office staff might still need to take. For example, website might still need to be updated with revised schedule, or instructors may still need to be notified of changes.
TABLES UTILIZED IN SCHEDULE AND SYNC WORKFLOW¶
Placeholder table for schedule data from css, filled when getcss.py is run.
Copy of css_sections table but with post-sync updates to reflect changed or assigned instructors. used by putcss.py.
Contains row with file_name = css.log which shows any messages from getcss or putcss.
Supplementary table when sync code is looking up rooms, capacities, etc for various reasons.
Used throughout the sync code to get detailed attributes of a course (course format and so on).
Each time sync is run, enrollment counts for each course are saved to this table. One count for all sections offered per course/semester.
tbl1_Instructors_ClassesTaught
This is one of the two main tables that hold schedule data. It is used throughout the sync code for many reasons.
Used as supplementary table throughout this process to cross reference instructor information (ie getting their netid and so on)
Used as supplementary table when info needed on a specific classroom (like when generating schedule html)
One of two main tables where schedule data is held
tbl1_schedule_inbound_from_css
When sync begins, data is copied from css_sections and reformatted and saved into this table in the way that the solutions code expects prior to doing all the sync comparisons. this is mostly a holdover from old sync code (prior to python/control center) where the bulk of comparisons were done with data assumed to be in this format, and doing this enabled us to hold off and avoid rewriting lots of things immediately The structure of this table more closely matches the tbl1_Schedule_Data table which is Solutions main table for schedule data.
Table that lists semesters solutions cares about, and current status code. Tells sync which semesters to bother with.
High level semester, year, timestamp and # changes flagged for each time sync is run (one to many with tbl1_sync_messages with id = tbl1_synd_messages.sync_id)
Messages that were generated from each syn (many to one with tbl1_sync_log where sync_id = tbl1_sync_log.id).