Monday, 11 May 2015

Looking back at Pre-Exam 2015 after some successful appeals

Last Thursday, "Appeal success" posted as comment to our blog Examiner's Report and Results Pre-Exam 2015: "It seems that appeals based on 15.2 and 17.3 have been successful and reversed by the Examination Board themselves without the need for the intervention of the [Disciplinary] Board of Appeal - an Appellant at our firm received a letter today and it looked like a standard letter."

Anonymous, Jessica and Onur confirmed to have received the same good news: Jessica shared the content of the letter with the blog readers:

"The Examination Board has considered your appeal well founded and has decided to rectify its decision. Your answers to statements 15.2 and 17.3 are justifiable and an evaluation of your work on this basis would lead to your passing the pre-examination 2015.

The decision communicated to you with letter of 27 March 2015 stating your having failed the pre-examination 2015 is therefore withdrawn by the examination board.

Your answer paper has been awarded the following marks: 70

Based on the marks above and pursuant to Rule 6(2) IPREE, the examination board has decided to award the grade PASS.

I also take this opportunity to indicate that application to the main examination (papers A, B, C, D) of the EQE 2016 will start on 14 July 2015.

Should you have already enrolled to the pre-examination 2016, your application will be deemed null and void and the fees paid will be reimbursed.

The appeal fees will be reimbursed as soon as possible."


An extensive discussion can be found on IPKats blog "A day late and a dollar short: EQE appeal outcome favours the brave". Both that blog post by IPKat itself as well as the comments posted doubt the suitability of the True-False or any other multiple choice format for the Pre-Exam. Also, IPKat does not understand how it is possible that the guinea pig tests and further quality systems were not able to prevent the problem to occur, while the day after the exam, the discussion on the DeltaPatents' blog showed that there was a real issue with the cardboard. 

IPKat indicated "that one part of the problem may be haste. The emphasis on getting results out quickly is fine until the integrity of the exam starts to suffer, and from the outside it looks as if the committee, Board or Secretariat could have done with being allowed a little more time with the announcement of this year's marks. The  Supervisory Board might need to look at the time given to get the results right." In this context, it may be worthwhile to recall that -on last year's meeting between the Committees and the tutors- the Pre-Exam Committee did not accept the offer of a group of tutors to review the answers before making them public and mefore the marking, as there was no room for interference of tutors (see epi information 4/2014, page 138 ff , 3rd bullet in right column on page 145).

Further, quite harsh comments were posted to the IPKat blog to the 2015 Pre-Exam, as now three (claims analysis) statements are qualified as not ambiguous (6 marks) and even more statements are subject to further appeals...  (some of them having a high chance of success, in my view, 5.2, 16.3, and 20.4). Last year it was one (legal) statement (10.4) where appeal was succesful -- and where the decision was reversed for all 15 candidates that  originally failed whereas only 5 of them appealed. 

Although I do not share all of the statements made there, it seems that the successful appeals, the blog posts, the comments and the unrest caused to several tens of candidates (probably  about half of the 59 candidates that scored 66-69 marks, see here) seems to call for some improvements.
It is not only a long period of unrest and uncertainly for candidates, but -with these uncertainties as to (un)ambiguities- it is also for us tutors difficult to defend the pre-exam's usefulness and fairness, and it is also not really our favourite act to defend  answers that deviate from the Examiner's Report answers and to hereby show that the Exam Committee was wrong...


14 comments:

  1. Roel

    Just a follow up from the IPKat side: We've received an anonymous comment this morning which indicates that even candidates who did not appeal, but who were affected by questions 15.2 and 17.3, have had their result revised from Fail to Pass where appropriate. I've edited my post in light of that news, and hopefully others here can confirm that they received such a letter even without having appealed. While the Exam Board looks to have fixed the problems, I still think that the approach to the pre-exam could do with being reviewed to make it more bulletproof and to catch these problems before the results are communicated to the public. Like you I don't want to criticise the Committees as I know they put in a huge amount of work with little recognition, and they have to work to a schedule and exam format over which they have little say.

    ReplyDelete
    Replies
    1. Hi David,

      I also do not want to criticize the Committees - it is very very difficult to make fully unambiguous True/False questions without giving away the answer. What I am rather critical about however, is that the Commitee indicated last year at the Tutor's meeting that they were not willing to accept both True and False for certain statements as correct - whereas they should, and gladly also actually did.
      It did surprise me this year that, even though it became clear from the moment on that we posted our answers the day after the Pre-Exam and that tutors as well as candidates immediately identified the "cardboard issue", that there was no review -at least no public announcement as to that- of the issue by the Committees. Some candidates did the effort to appeal, and for those reversing the decision is correct. There is however no legal basis for reverting the decision for those that did not appeal - and that did not have the risk to loose the appeal fee is their appeal would not be considered justified. (And what would have happened if nobody appealed?) All appeals and the other reversals would however not be necessary if the Committee would have reviewed their answers -as they did for 20.2- without any appeals before they started marking, or -as they do for the main exam papers- after a first marking of part of the papers to detect possible problem areas change/correct the marking table.
      I hope for the benefit of the system and for the credibility/fairness of the Pre-Exam, that carefulness will become dominant over the currently used speed of sending results letters where tens of results letters need to be reversed two-and-a-half months later... You also indicated that on the IPKat blog, and in the comment above. There is no need at all to communicate results within 3 weeks if that leaves tens of candidates into an even more uncertain position than if they did not get the results -- and that may also have let to some candidates getting a pass whereas they should have failed in case the original official answer was wrong...
      Thx again for your comment.

      By the way, would it under UK Common Law be OK that a court reverses its decision at own motion, without it being appealed (in civil cases)?

      Delete
  2. David, I'm wondering a bit whether that is really what he wrote. "So apparently they did send a letter to everyone that got affected by the appeal." seems at least ambiguous as to whether he appealed or not...

    Revising the results after notification of the decision without a pending appeal seems illegal to me. I realise the interests of third parties might not really be at stake here, though ;-) (that might be a bit different if this was about the EQE itself).

    ReplyDelete
  3. I think we have only one comment "DidNotAppeal11 May 2015 at 11:07" stating a revision without appeal. Would be nice if it was confirmed more clearly. For me, it would take away my main concern about fairness. And a positive post would then be deserved.

    ReplyDelete
    Replies
    1. It would be fair on one hand, but not on the other. It is not fair to those who did go through the effort and invested their time to file the appeal -with the risk of loosing 1200 euro-. It nobody would be willing to do so, would the decisions also have been changed? The Committee, Examination Board or Supervisory Board could have been praised if they did not only do interlocutory revision after appeals were filed, but if they would have done it before the decisions were send out -- now the review relies on the Dutch saying "iemand moet de hete kastanjes uit het vuur halen" (someone needs to burn his fingers on the hot issue), whereas the issue and the justified solution were already clear two days after the exam and three weeks before the results were made public!
      So, yes, a good move forward to allow interlocutory revision in stead of sending it to the Disciplinary Board, but further improvements would be appropriate to be 100% fair, at least in my personal view.

      Delete
  4. I know two people who was only wrong in one of the questions and at least neither of them got any letter so far. I know its a different case, but I don't know what would be most fair; to only correct for those who got both wrong or for everyone.

    /Jessica

    ReplyDelete
    Replies
    1. I think also those who only got one wrong should be corrected if they failed based on that:

      a good reason to have only one wrong is that the candidate actually saw it was bnot 100% clear in the paper: you can than choose as a candidate if two statements are affected:
      a) answer both based on one interpretation -- you play all or nothing them: you either get 4 (if as expected) or 0 (if not as expected) marks (assuming the other statements of the sets of 4 are correct); or
      b) do not want to risk to get 0 insted of 4, and play save to get 2 marks (and not get 4, but also not 0)
      I know that at least one candidate took the last approach! That candidate certainly understand the relevant topic!!!

      Delete
    2. Wow, it could get very interesting if someone got one wrong, failed because of that, did file an appeal, and now did not get his results corrected because the "very smart" Examination Board figured that this person clearly had no clue, so did not deserve extra marks.

      The Disciplinary Board of Appeal might use less than nice words.

      Delete
    3. In D 3/14, r. 9 and r.17, they already used quite clear and unambiguous statements as to this - even though still in quite nice words- ...:

      9. Pursuant to Article 24(3) REE, the department whose decision was contested (in the present case the Examination Board) must rectify its decision if it considers the relevant requirements to be fulfilled. This also means that the Examination Board is obliged to assess carefully whether or not these requirements are met before deciding to grant or refuse rectification and, in the latter case, referring the matter to the board of appeal (D 38/05 of 17 January 2007, point 3 of the Reasons; D 4/06 of 29. November 2006, point 3 of the Reasons). In clear and unequivocal cases, rectification is a quick and simple way of cancelling flawed decisions and spares the parties the cost, in time and money, of appeal proceedings. Rectification is thus in the public interest and in particular in the interest of the appellant (D 38/05 of 17 January 2007, point 2 of the Reasons; D 4/06 of 29. November 2006, point 2 of the Reasons). Having regard to the obvious discrepancy between the question underlying statement 10.4 and the expected answer according to the Examiner's Report, rectification (Article 24(3) REE) was warranted in the present case, especially in view of the fact that candidates who apply to sit the European qualifying examination must first pass the pre-examination (Article 11(7), last sentence, REE).

      17. In the judgement of the appeal board, the fact that the Examination Board did not rectify the flawed decision, even though the discrepancy between the question underlying statement 10.4 and the expected answer according to the Examiner's Report was comprehensively substantiated in the appellant's grounds of appeal, must also be taken into account. The Examination Board thus burdened the appellant with appeal proceedings.

      ...

      Delete
  5. I don't believe everyone who answered one or both questions "wrong" would be expecting a letter. The Board is only changing the outcome (pass or fail) and not the mark itself as originally allocated by the exam committee, or at least that's my reading. So if the extra couple of marks would not bring you over the pass line, then no change of result is warranted and I think no letter would be expected.

    ReplyDelete
    Replies
    1. I fully agree with David - that is at least what happened last year when the results were changed of 15 candidates: 5 that appealed and 10 that did not appeal but got from fail to pass by the review of statement 10.4.
      Actually, you can in my view also only appeal the DECISION, and not the SCORE - the decision is actually to award a pass or a fail -- the ground for the decision are the score (and how the score relates to the pass level). An appeal which does can only change the score but not the pass/fail decision would in my view not be admissible, as the appellant is not adversely affected if the decision stand but only the grounds are changed.

      Delete
    2. The EQE webpages do not give any indication (yet) about statement 15.2 and 17.3 being considered correctly answered with both T and F, not in the Examiner's report, not in a separate message, not in the Results file (that carries the date of 20.03.2015).
      Can be expect the Committee to be clear on this?

      M

      Delete
  6. So, in the end do we know how the marks were changed- only for the appealing candidates or for all candidates who initially had a FAIL and would have a PASS if their answer to 15.2 and/or 17.3 was deemed correct?

    ReplyDelete
  7. As is clear from the post of 3 June and the Addendum to the Examiner's report, all candidates wo would had passed but for the cardboard issue had their marks upgraded.

    ReplyDelete