Monday, March 20, 2023 ChatGTP-4's attempt at parts 3 & 4 of the Pre-Exam 2023

We also let ChatGPT provide answers to parts 3 & 4 of the Pre-Exam.

Some practical difficulties were that only plain text could be entered, which means that in part 3, the figures couldn't be provided, and that the underlining when indicating amendments was not present.

If one would consider our answers (see seperate posts, which are not necessarily correct!!) as the intended solution and applied the Pre-Exam's marking scheme (all correct: 5 points, 1 wrong: 3 points, etc), ChatGTP achieved 16/50 points which is well below the required passing grade of 35 (normalized from 70 for the entire exam to 35 for parts 3 & 4) and rather a score associated with mere guessing.

Generally speaking, ChatGPT seems to be able to answer basic novelty and scope questions reasonably well but fails at more specific topics such as ranges. Also, ChatGPT appearantly disagrees with the EPO's relatively strict approach to Art. 123(2) and considers most of the amendments as supported. Perhaps this reveals ChatGTP's US-origin?

See below for ChatGPT's answers and short reasoning (red marked where ChatGPT's answer deviated from ours):

Monday, March 20, 2023 ChatGTP-4 finds the pre-exam legal part quite challenging.

From the ChatGTP-4 paper, we understand that its ability to pass exams has been greatly increased. For example, "on a simulated bar exam, GPT-4 achieves a score that falls in the top 10% of test takers. This contrasts with GPT-3.5, which scores in the bottom 10%.", see [1]. Naturally, we were curious to see how the AI would do on the EQE. The pre-exam legal part seems to be the most accessible for an AI, so this is the part we tried. 

We did two runs, one with a short prompt, and one with a long prompt. Both prompts explain the exam's requirements and ask for step-by-step reasoning. The long prompt also contains a question from last year's exam with an example of the required step-by-step legal reasoning. The prompt had to be very clear the program always needs to answer True or False, otherwise you get a lot of answers explaining why the available information is insufficient to make a decision. 

Saturday, March 18, 2023 Pre-Exam 2023 (part 4): erasable inks - our answers

The second claims part was about erasable inks.

“This invention provides erasable inks that provide good writing performance when used in writing instruments such as pens. The erasable inks are water-based inks that have a short "waiting time", i.e. inks that can be erased soon after they are applied on a surface made, for instance, of paper. Because the water-based inks are based on using water as a solvent rather than on using other solvents, the erasable inks of the invention are relatively non-toxic and odour-free.” [002] of the description of the client’s application)

Our answers to this second claims analysis part are given below. We have numbered the questions and statements for easy reference in the discussion.

Saturday, March 18, 2023 Pre-Exam 2023 (part 3): cleaning utensils - our answers

The first claims part was about cleaning utensils, in particular cleaning cloths.

Our answers to this first claims analysis part are given below. We have numbered the questions and statements for easy reference in the discussion.

Friday, March 17, 2023 Pre-Exam 2023 (part 1 & 2): our answers to the legal questions

This was the third online -EQE with the Pre-Exam being held in four parts, and with the questions almost fully being taken from the screen (only the description of the application and the prior-art documents in the claims analysis parts were printable; also no calendars were provided).

Our answers to the legal part are given below. We have numbered the questions and statements for easy reference in the discussion.

Friday, March 17, 2023 Pre-Exam 2023: first impressions?

 To all who sat the Pre-Exam today:

What are your first impressions to this year's Pre-Exam? 

Any general or specific comments?

Were the legal topics well balanced? Was the balance between EPC and PCT right for you? Were recent changes and stable legal provisions tested in the right balance for you? Which of the legal questions did you consider particularly difficult, and which relatively 'easy'?

Were the various aspects of claims analysis well balanced? How did you deal with the situation that only part of the paper (only the prior art) could be printed? 
(How) did you use the available online legal texts (EPO legal texts incl Guidelines, EPC Articles and Rules, RFees; PCT Applicant's Guide, PCT Treaty Articles and Rules)?

What is your expectation of the pass rate and the average score? How many marks do you expect to have scored in the legal part, in the claims analysis, and for the whole paper?

Were you able to finish each of the legal parts of the exam in the 70 minutes available for the appropriate part? And each of the claims analysis parts? 

Did you experience any technical difficulties during the exam? Which? How and how fast were they solved? Did you use the search functions? Did you use annotation, sticky notes and other Wiseflow tools?

How did this year's paper compare to the earlier pre-exams w.r.t. the pre-exam as a whole, w.r.t. the legal part and w.r.t. the claims analysis part?

Friday, March 17, 2023 Our Pre-Exam 2023 blogs will open after the exam, 17 March 2023, 16:10

Good luck with the Pre-Exam 2023!

Our EQE blogs will be open for your comments and opinions w.r.t. the Pre-Exam, A, B, C and D shortly after the exams. We will post our (provisional) answers to the various papers shortly after the exam - for the Pre-Exam, we will use separate blog posts for the legal part and for the claims analysis part, and one for first impressions. 

Do not post any comments as to the merits of the answers of a certain exam paper/flow on the blogs while an exam/flow is still ongoing. Also, do not post the invigilator password or anything else that may be considered the breach of the exam regulations, instructions to the candidates, code of conducts, etc (see, e.g.,  EQE homepage, EQE notices, EQE online website, MyEQE, and the emails from the EQE secretariat).

All candidates, as well as tutors who helped candidates prepare for EQE 2023, are invited to contribute to the discussions on our EQE blogs! You can post your comments in English, French or German. You are invited to post your comments under your real name, but it is also possible to use a nickname if you wish to hide your identify.

The DeltaPatents team

NB: you can not comment to this blog post; comments will be accepted from a new blog post "First impressions" as of 16:10, i.e., which will become available after the Pre-Exam has finished.