Competition Rules

The CHART-Infographics competition is open for participants from both industry and academia. The competition is open for both students and professionals who want to make a contribution to the field of chart recognition. Below we describe the rules for our competition.

• Each participant team can include up to a maximum of 10 people from one or more affiliations. For the sake of fairness to smaller research groups, we will not allow bigger research groups to participate as a single team.
• One person can only participate on one team. Mentors included. No exceptions.
• Winners teams will have certificates listing the names of their members in the exact format and order as they were registered.
• We will only allow registrations to be updated until the date of the release of the testing dataset. Changes allowed before this date include adding or removing members, correcting typos on names, as well as updating the order of the team members which will be used for the certificate in the case of winners.
• Participants can register to participate on just individual tasks and sub-tasks. However, winners will be selected based on their task coverage. We will be using a scoring system as follows:
  • For each task/sub-task, we will rank all participant teams using the average and standard deviation of the corresponding task metric. Two or more teams will be considered to be technically tied on the same rank if the difference of their means is not statistically significant.
  • Each team will get points based on their rank on each task: rank 1 gets 5 points, rank 2 gets 4 points, rank 3 gets 3 points, and rank 4 and below get 1 point for participation. Not submitting results for the task gets a score of 0.
  • The total scores for all tasks will be added up for each participant team to obtain a single final score used to pick the overall winning team.
• Participants will be allowed to use their own datasets for training. However, they must notify the organizers about this.
• Submissions will be done via e-mail. Participants will be required to submit the results of their systems in the expected JSON format. Participants will be able to beta test their system using testing sets and evaluation tools from the previous competition. This will be helpful in making sure that their system can produce outputs in the right formats. We might not consider submissions which do not conform to the correct file format.
• No CODE submission will be required.
• Participants must provide a description of the methods used to produce the results submitted. In the final competition paper, we will summarize these descriptions when we describe the submitted systems. If external datasets were used, participants will also have to provide a full description of these. We reserve the right to disqualify submissions which do not provide a sufficiently detailed description of their system.
• To be fair to all participants, any deadline extensions given will apply to all participants, not just to individual research groups who might request them.
• Our data and tools will be made publicly available after the competition.