New Home
The Web-CAT Development log has moved to a new location on Stephen Edwards' home page. Go there for current information.
The information on this page remains for historical purposes only.
Old Materials
This page contains a brief log of recent changes to Web-CAT. Many small changes aren't worthy of separate announcements, but they are posted here for those who want to know a bit more about the "inner workings" of Web-CAT development. This page isn't a real blog either, since it isn't doesn't support the kind of cross-linking one would expect of a true blog. However, it feels a bit like a blog because of the nature of the content.
Recent Web-CAT Development Activities
[2005-11-29] Graphing and visualization After some long hours over Thanksgiving break, I was finally able to bring up the graphing and visualization support that has been cooking in the background since spring. Now students can see a few different graphical views of their code, and instructors can see a bit more. Look for the little graph icon under "Actions" in lists of assignments, or use the Graphs subtab under the Results tab.
[2005-11-27] A shortcut to submit again Based on student requests, I have added a "Submit again" button to the results page that students see showing their assignment feedback. We had one of those way back, but it disappeared in the UI redesign last winter. Now it is back.
[2005-11-27] Graphing score distributions
A couple of students in a grad course on educational tools designed some simple graphing views for Web-CAT, both for students and for instructors. I spent some time over our Thanksgiving break designing the appropriate incremental statistics gathering code so that it would be relatively cheap to show basic score distributions. Then I added some simple HTML/CSS-based histograms that take up very little space, and now the main list of assignments for students show score distribution information, right on the front page. Now I'll work on adding a few more sophisticated views using JFreeChart (http://www.jfree.org).
[2005-11-26] Better guessing of due dates for new assignments
ManuelPerez and I have had several discussions about the way Web-CAT picks default dates when you create new assignments. The rule is simple: it picks 11:59pm two weeks from today. Unfortunately, this is always the wrong choice ;-). After thinking about the problem, I devised some heuristics and finally implemented them. Web-CAT tries to make a smarter guess based on other assignment dates you have given in the past. We'll see if instructors like it.
[2005-11-26] A better date/time picker
Instructors enter/edit due dates for assignments using a simple text field that is parsed using a date parser. However, the parser is very brittle and there isn't much data validation, so it can be easy to make mistakes (like inserting an extra space, for example). Plus, it is difficult to pick dates without a calendar for reference. I'd wanted to fix this problem for a long time, but it hasn't been too high on the priority list. Well, I happened to be reading about a new javascript date picker, so I started digging around and found one that would work nicely for us: Dynarch's DHTML calendar. I added it to the assignment editing page and added a few tweaks, and it works great! One nice future benefit: this calendar component makes it easy to highlight specific dates in the pop-up calendar. In the future, I'll highlight dates when other assignments are due so instructors will have a visual warning when picking due dates.
[2005-11-25] Fixed bug in saving script parameter choices
Several people, including ManuelPerez and GrantBraught, have pointed out a bug where Web-CAT sometimes fails to correctly save configuration parameters for grading scripts. This is most often visible when the instructor is uploading a new file to use for one of the parameters. I thought the bug was fixed a while back, but it wasn't. Now it is!
[2005-11-22] Automatic database schema updates
For a long time, I've been interested in implementing a database schema updating scheme similar to the one used by Moodle. When you install a new version of Moodle, it checks the database on startup and automatically "upgrades" the database schema, adding any new columns or tables needed by the latest features. In fact, the first time you run it, it will automatically create the entire database structure itself. Further, because Moodle uses a modular, plug-in architecture, separate modules are responsible for maintaining their own tables, and all modules use this same scheme to automatically apply improvements and upgrades to their own pieces of the database.
I'd been thinking about doing something similar for Web-CAT, and ran across an article by James Seigel and Christian Pekeler on http://www.stepwise.com presenting a similar design for WebObjects programs. I also found a modified version of the same code by Helge Staedtler on http://wocode.com. These were excellent sources, although I wanted to be able to support an arbitrary number of independent subsystems with their own separate groups of tables. I adapted the code in these examples and created a new framework for automatically applying subsystem-oriented database updates. The code is very slick, and works like a charm. This addition, although it was a bit of a side tour, provides three main benefits. First, whenever I modify any table definitions (like I am about to in order to support some graphing features), the application will automatically update the database. Any other sites that use our WAR-packaged application will receive the updates for free, without any manual database jockeying needed. Second, since I added baseline updates to create tables that are missing, the application will auto-create all the necessary database tables for a fresh installation, again without requiring any manual effort. Third, I added a bit of code so that when tables are auto-created, they are also auto-populated with some defaults, so an administration account and a default authentication scheme are pre-populated automatically. This has moved Web-CAT a long way toward a much simpler installation process, with many fewer steps to get the app going. Now to try it out by using it to add some new features ...
[2005-11-17] Resolving conflicts between instructor and student test classes
To prevent conflicts, we've always advised instructors to give unusual names to their test classes so that there is no possibility of a conflict between student and instructor classes. Long ago, the Java grading script's ANT build file assumed this. However, this usually caused problems when instructors submitted their own reference solutions when trying to test a new assignment: they would see a zero score because of the conflict. I thought I'd fixed that problem before (see the 2005-09-29 blog entry), but the solution resulted in a strange behavior: if a reference test class and a student test class conflicted, the reference test class is the one that would be used. This didn't cause trouble for instructors submitting their own reference tests, but if a student was unlucky enough to use the same name for one of his or her own test classes, it would never be executed. Manas Tungare found this out on one of ManuelPerez's assignments recently. So now I fixed the fix: instructors can name their test classes anything they want, and so can students, and no conflicts ever occur.
[2005-11-16] C++ code coverage highlighting errors
It looks like when I added support for BullseyeCoverage to the C++ grading script, I accidentally introduced a few bugs that caused generation of invalid xhtml when merging coverage messages with pretty-printed code listings. In some cases these errors were preventing students from seeing their listings, and in others they were interfering with course staff performing grading tasks. I fixed the errors today.
[2005-11-15] Automatic reloading of script properties Grading scripts that plug into Web-CAT usually carry along a set of property definitions that determine exactly how the instructor can set parameters for the script, as well as other non-editable properties used by Web-CAT. If you upload a script, this property definition is parsed automatically. Further, if you edit the configuration file or upload a replacement through Web-CAT's web interface, the new version is automatically reparsed as well. However, the informational date shown on the script's edit page is the date it was originally uploaded, no matter what happens in the mean time. As a result, I modified the script code so that each time the config file is reparsed, the modification date shown for the script is updated. Further, the grading engine will automatically reparse the config file (and update default settings, etc.) if it detects that the config file on disk has changed since it was last processed. This check is performed automatically each time the script is executed by the grading engine. The goal is to eventually allow Web-CAT to automatically download updated versions of predefined scripts from an internet-based repository, so that it automatically picks up its own upgrades whenever they are published.
[2005-11-10] A better way to specify points
David McPherson registered a complaint today--actually, a long-standing one about the way instructors specify point totals for assignments. For a long time, instructors have had three text boxes to fill in: one for the total number of points for an assignment as a whole, one for the number of points out of this total that a TA or instructor hand-grades, and one for the number of points out of the total that are auto-graded using static analysis tools like PMD and Checkstyle. There was no slot to specify the number of points for correctness/testing, since that value can be derived from the others. Needless to say, this has caused some degree of confusion. Fixing the problem has been on the to-do list for ages. Today, I finally redid that page, so instructors specify the number of points allocated to manual grading, static analysis tools and correctness/testing. Some javascript on the page dynamically calculates and displays the total points available as edits are made in the browser.
[2005-11-9] More C++ signal handling improvements
Whew ... it has been a long time since my last update. Besides the normal assortment of small fixes, the biggest change lately is a significant revision to the CxxTest-related signal handling code for running student tests. The new revisions (which are running on the VT Web-CAT server) include the following features for students who are using CxxTest on their own machines via our Eclipse plug-in:
- Some errors in tracking internal calls within Memwatch.cpp have been fixed.
- Signal handling now works properly for all failures in setUp() and tearDown(), so such failures are treated properly as test case failures.
Assertions now no longer halt the test run, and are treated as proper test case failures, with assertion messages showing up in the CxxTest view correctly (like segfaults did before).
Memory violations (like double deletes, deletion of non-heap pointers, deletion of memory blocks that have been corrupted, out of memory errors, etc.) now all cause appropriate signals that result in test case failures and appropriate diagnostics in the CxxTest view. Similarly, on the Web-CAT side, such errors count as true test case failures, rather than just diagnostic messages.
- Assertion failures and memory violation support also supports easier debugging--just run the thing under gdb, and the debugger will halt as soon as the offending condition arises with a full, inspectable stack trace so students can find the offending code.
- The code now compiles under gcc 4.x.