Atutor.no

A module we've (that is Håvard) been working on for a long time is now ready for release to the community! We called it Certify, it is mighty fine, and this is what it does:

Certify will let you add a certificate to a course. The certificate can contain one or more tests, and when a user has reached the pass score / pass percent for the test(s) bundled in the certificate, the user is allowed to download a pdf certificate.

Before the user has reaced the pass score he or she will see a progress bar under the "Certificates" tab, showing how far he or she is on the way to obtaining the certificate.

An instructor (or privileged user) can manage certificates, which means adding them, adding tests to them, and see student status for a given certificate.

Pictures are however more telling than words.

Here is certify for a student

This shows the certificate name, a description, and a progress bar showing how far the student is in obtaining the pass scores for the 8 tests in this course.

In the picture above I am 21% on my way to obtain the pass percent of 80% in all the 8 tests this certificate consist of.

In the picture below I have reached pass percent for all tests in this certificate. The certificate status is then 100%, and I am allowed to downdload a pdf certificate.

A certificate template can be created and uploaded by the instructor. If not a standard certificate is used (taken from the other Atutor Certificates module by Cindy).

Here's a screenshot of the certificate I got from the link above:

Yes, it is sweet!

Certify for an instructor

Here we are at manage -> certify, which shows the certificates in this course (there can be several certificates per course, allthough I imagine most people will only use one).

If we click on the "edit tests" button we get this:

This shows a list of all the tests that are found in this course, and lets you add as many of them as you wish to the certificate. Note that these tests must have a pass score or pass percent set. Also note that Certify only acts as a container. The pass scores or pass percents must be set in the tests themselves, and certify will only do the calculation for you.

The next screenshot shows what you will see when you click the "Student status" button:

..a sorted list of the students results. Note that certify only cares about the best result a student has in a test. That means a student can improve his score in a test (where more takes are allowed), but not the other way around.

That is pretty much it! We are using it in production today, and you can have a look at it in action if you register for the Medical Peace Work course

We have found it to be stable for production use, but there are still some bugs, minor issues and areas for improvement left in it, mainly for admins of it:

• You can upload a pdf template for each certificate, but there is no way to see which template you are using just yet, and
• If you upload a new template you will have to manually delete the old one form the server, as well as any cached files (named typically cert*.pdf (where * is a wildcard). The files are found within the ATutor content folder, in the "certify" folder.
• You will need pdftk installed at your (Linux) server! A server admin must typically do this.
• And you must manually set the path to pdftk in line 100 in download_certificate.php. Here's the line: $exec = '/usr/bin/pdftk '.$template.' fill_form '.$filename.' output '.$filebase.'pdf flatten';, as you can see here our path to pdftk is "/usr/bin/pdftk ".
• There are some tweaks that should be done in the GUI as a radio button isn't selected if you click the table row it belongs to. You will have to hit the actual button!
• And probably some more too

We're adding it here first, and at atutor.ca in a few days - get it while it's hot, give it a go, and give us feedback!

I've been posting several things we wish to adjust, change or develop in ATutor lately, and here's one more!

One of the things we get the most negative feedback and support questions on, is how assignments work, both from students and our instructors.

Today you deliver assignments from the "my files" area of the file storage. There are some good things about this, specifically that you can use the "my files" are to store your assignments, keep revisions, and work on them there till they are done, then deliver them with a click or two. Also, creating a new assignment for instructors works just fine!

Sadly there are also many problems with this.

For students:

• Most of our students don't use "my files" to store these files, so delivering assignments through it becomes cumbersome. You first have to find and upload to "my files", select files from there, click deliver, then choose which assignment to deliver, then actually deliver.
• You need to activate "file storage" even though it isn't used for anything else than assignments.
• The students have nowhere to see which assignments a course has, or when they are due before they actually select a file and click "hand in". Then you get a list of the different assignments with their due dates.
• Feedback on the assignments are delivered as a comment to their files, so they need to manually check back here to get feedback.

For instructors, which is where we get the most complaints:

• First and foremost, every time a course uses assignments we get this complaint: there are no way for an instructor to see who has delivered and who has not. When chosing assignments -> submissions they get a list of folders named after the users username. They then have to manually click each folder to see if there are files handed in.
• Earlier there was no way to give feedback from here either, but now you can do this via the comments. However, it is still cumbersome to see who you have commented to, and who you haven't

Main issue

I believe the main issue here simply is that the file storage is designed to be a tool to store files, not a place to hand in assignments. It works great for what it is originally designed for, but not optimal for what it is tweaked into also doing.

I am personally partly responsible for this, as we partly specified this functinality years ago, but that doesn't mean we shouldn't look at it again now, after we have gotten this feedback and experience with how students and instructors use it.

For me the solution is to move it out from the file storage and do one of the following with it. Which of these is better is up for debate:

1. file uploads as separate question type to be used in tests.
2. assignment as own tool.

File uploads as question type to be used in tests.

In this scenario we would use the test tool as it is today, but assignments would be handled through a specific question type that would allow you to upload files. The typical scenario for us would be to create a test with one / two questions of the file upload kind.

The good thing with this is that you would have all the functionality that is in the test tool today (title, description, availability, number of possible tries, when to release results, submission lists etc). It would also work with our certificate module out of the box, an all in all it seems to me simpler to develop and maintain.

You would no longer manage these with manage -> assignments, but via the test tool.

The bad thing is that assignments and tests are different things, and this might be confusing. You would also need to name the tool (and thus the tab) "Tests & Surveys & Assignments" (in Norwegian "Tester, spørreundersøkelser og innleveringer") which undoubtely is a very long title for a menu item.

Assignments as it's own tool

In this scenario you would still manage the assigments as today, but they would be available to the students through its own tool / tab, named "Assignments". We would have to develop more, like a student front end, some way to see submitted assignments in the assignment module and etc.

What's better?

I think I personally lean more to one of the solutions above, but I would love to get some more feedback here - fire away in the comments!

We are moving more into automated tests where the system takes care of what the pass score or pass percent is, and then gives the user a certificate to download (more on our new Certify module coming soon).

In this process we have discovered a problem with the "multiple answer" question type, that can be best illustrated with this screen shot:

ATutor tells the user how many points are available, and in this case there are 10 questions with 10 point available, which for the not-very-observant student means that all alternatives are right. The problem is still there if there was 5 points available: the student could check all alternatives and reach maximum score for that question.

The underlying problem here is that you can only give point per question, not per question alternative, which means that you need a human beeing to correct all tests and this automatic correcting is not possible.

To solve this I believe we need to be able to set points per alternative, and also to set negative points (so you can't just check all and get away with max score).

let's take a look at one question:

This is how I believe this question type must work. Now, this will create a new issue, which is that creating these questions will be harder, so we need to take that into account as well.

As this is pretty simple calculating I believe the system should do it by taking the point value set on a multiple answer question, and then divide it as positive and negative scores on each alternative, which if needed can be changed by the creator at a later time:

We are willing to do this job, but need feedback on how to best solve it! I also wonder if there will be any consequences for backups and restoring of tests, and the IMS QTI Question and Test Interoperability Support?

I am not in any way a programmer, but I'm slowly learning some php, and today I actually made something useful for ATutor that has been requested by several of our instructors!

The code now work just as it should, but I can see that it might be a smarter way of solving this, without all the redundant lines! Also, I have some other plans to further expand this before I will wrap it all up in a patch.

Problem is this

For tests you can set no pass score, a pass score in points or a pass percent. This is good, but when an instructor views the test submissions on a test it isn't reflected anywhere wether the student passed or failed. This is what the results table looks like:

Problem here is that the the student needs 15 points to pass so three of these student failed and three passed, but the table doesn't tell the instructor this. And quite naturally, many of them wish to see this information!

So I set out to change the tools/tests/results.php to fix this problem. Here's what I did:

Line 86 looks like this:

But we need to use the pass percent and pass score info from the database too, so I added these two values as variables for simplicity:

So far so good, now our page knows this info too and we can use them later!

Move down to lije 223 and we find the table row with the test result data:

(intendation is better, blame my editor here). Now, this presents the score if the questions in the test has scores, but we want it to also say if the student passed or not, and make it work both for pass scores and pass percents, so I changed it all too:

(Again, intendation is better) - now this actually works great, but as I'm repeating two chunks of pass / fail html code there must be a more elegant way of doing it, right?

Anyhow, the end result after these changes are:

I used existing graphics from ATutor so nothing new is added, and I made some styles that can be improved on, but atleast it works!

Further expansion

This solves some problems, but the submissions page can still be improved, and the first things I wish to do is add two more filters to it:

1. You need to be able to filter by pass or fail too, so an instructor can choose to only see the ones that passed a test or failed a test.
2. ..and more importaint: we need a way to hide redundant scores. The situation for many of our tests now is that the users can try many times to get it right, so you can easily get 300 submissions on a page for say 50 students. Now, the instructors are most often only interested in the 50 students, and wether they pass or fail, so we need a filter that only gets the best score a student has achieved, and then only displays that one result per student, making the instructor able to easily see which of his students passed or failed, without caring how many tries they have used.

Any ideas for this, and also other ways to improve this page are appreciated! I add the modified results.php page here so you can try it on your own install, but do take a backup!

One more patch aimed at improving usability of ATutor. It enables group deletion of test submissions by replacing radio buttons with checkboxes and adding a "Select/unselect all" checkbox.
Also it fixes minor bugs found in that area (I've noticed as much as seven of them).

The change has already been included into the trunk for the next release, 1.6.2, while this patch allows users to benefit from it right now.

See the patch here: http://atutor.no/contribs/group-deletion-test-submissions-plus-bugfixes