[cfe-dev] Using clang static analyser / clang-tidy for assignments grading?

[Nathan James via cfe-dev]

Hi Yahav,

It is certainly possible using the static analyser and clang tidy frameworks to implement most of that checking. In clang tidy there are dedicated checks for some of the restrictions you need. Others would be vary easy to write AST matches or preprocessor callbacks for.

However it's my personal opinion that this should not be used as a replacement for a human grading but more as an assistant. When patches are submitted for review here we have automatic checks that check validity and can reject obvious flaws but it is always up to a reviewer to have a look over before a go ahead is given.

If you do decide to go down the route of using clang to help out then definitely check out the code in the clang-tidy checks(static analyser not so much), have a look at the AST matches docs https://clang.llvm.org/docs/LibASTMatchersReference.html and have a play with clang-query.

Kind regards,
Nathan James.

________________________________
From: cfe-dev <cfe-dev-bounces at lists.llvm.org> on behalf of Yahav Bar via cfe-dev <cfe-dev at lists.llvm.org>
Sent: Friday, 28 February 2020, 16:29
To: cfe-dev at lists.llvm.org
Subject: [cfe-dev] Using clang static analyser / clang-tidy for assignments grading?

Hi everyone!
I hope that I'm querying the correct mailing list about my question.

I work as a teaching assistant at the Hebrew University, teaching C and C++. As part of our course, we ask the students to submit C and C++ exercises which we grade (both manually and automatically).

When grading students exercises, we check the validity of their code. For example,

  *   We check if the students didn't forget to use include safe-guard;
  *   If the students used non-safe functions, which we consider forbidden to use (in addition, sometimes we explicitly tell students not to use set of predefined functions or C++ classes, as the exercises ask them to implement this set of functions).
  *   If the students didn't include a forbidden header;
  *   If the students didn't use a #pragma statement to bypass our compilation instructions;
  *   C++: If the students remembered to use "const" when required, and to pass parameters by reference when needed;
  *   C++: that the students returned lvalue when needed and rvalue when needed etc.
  *   C++: When writing an iterator, if it was implemented correctly (a.k.a, according to Input/Output/Forward/Bidirectional/Random Access iterator rules + using iterator traits).

Right now these tests are being done by a human. As our classes formed from 300 to 600 students (next semester we'll have 650...) it'll be really hard and non-efficient to do it by hand. Thus I thought it might be a good idea to automate these checks too.

Initially, I thought to write that with Python using ANTLr (creating an AST for both the Preprocessing stage and the C stage and just iterating over them), but at the middle of programming, I came across the Static Analyser API of Clang and thought I should switch to it, as it seems very mature and well fit for our needs.

Before diving deeply into clang, I'd love to hear from you, who have experience in clang and LLVM dev, if I'm on the right track, and can actually achieve my goal using clang, or I should stick with my previous attempt. Our end goal is a program that we can run, send the path to the student exercise, and get the errors that she had so that we can reduce points accordingly.

Thank you very much!
Yahav.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20200229/3d727567/attachment.html>