21st Annual (2024) Humies Call for Entries

For Human-Competitive Results – Produced by Genetic and Evolutionary Computation

To be held as part of:

Genetic and Evolutionary Computation Conference (GECCO)
July 14-18, 2024 (Sunday - Thursday)

Melbourne, Australia (Hybrid)

Entries are hereby solicited for awards totaling $10,000 for human-competitive results that have been produced by any form of genetic and evolutionary computation (including, but not limited to genetic algorithms, genetic programming, evolution strategies, evolutionary programming, learning classifier systems, grammatical evolution, gene expression programming, differential evolution, genetic improvement, etc.) and that have been published in the open, reviewed literature between the deadline for the previous competition and the deadline for the current competition.

The competition will be held as part of the Genetic and Evolutionary Computation (GECCO) conference operated by the Association for Computing Machinery (ACM) Special Interest Group (SIG) on Genetic and Evolutionary Computation (SIGEVO). GECCO will be held in a hybrid manner, so either in-person or virtual participation will be possible--watch the GECCO website (see https://gecco-2024.sigevo.org/HomePage) for details.  Humies entrants selected as finalists may present either in person, or virtually by submission of a 10-minute video presentation.  Video entries and, if possible, recordings of in-person presentations, will subsequently be made available to the public on the Human-Competitive.org website, the GECCO website, and posting on YouTube.

The winners of the awards will be announced during the GECCO conference. 

This information is also available at the GECCO-2024 website, under the "Calls" dropdown at top of the main page, which takes you to www.human-competitive.org/call-for-entries).    At least one author of the submission must be registered for GECCO, although there is no requirement to present a paper there--only the in-person or video presentation of the Humies entry is required.

Important Dates:

  • Friday May 31, 2024 — Deadline for entries (consisting of one TEXT file, PDF files for one or more papers, and possible "in press" documentation (explained below)). Please send entries to goodman at msu dot edu  
  • Friday June 14, 2024 — Finalists will be notified by e-mail 
  • Friday, June 28, 2024 — Any finalists who will not be in Melbourne to present in person must submit a 10-minute video presentation (or the link and instructions for downloading the presentation, NOT a YouTube link) to goodman at msu dot edu. Finalists who will present in person must submit a copy of their (PowerPoint?) slides for the advance use of the judges, to goodman at msu dot edu.
  • July 14-18, 2024 (Sunday - Thursday) — GECCO conference (the schedule for the Humies session is not yet final, so please check the GECCO program as it is updated for the time of the Humies session, likely on Tuesday or Wednesday)
  • (Check GECCO schedule for posting) — Presentation session, where either a live presentation will be given or the the 10-minute videos will be shown (and later made available for viewing).
  • Thursday, July 18, 2024 — Announcement of awards at the plenary session of the GECCO conference  

Judging Committee:

  • Erik Goodman
  • Una-May O'Reilly
  • Stephanie Forrest
  • Wolfgang Banzhaf
  • Darrell Whitley
  • Lee Spector

PUBLICITY CHAIR: William Langdon

Call For Entries

Techniques of genetic and evolutionary computation are being increasingly applied to difficult real-world problems — often yielding results that are not merely academically interesting, but competitive with the work done by creative and inventive humans. Starting at the Genetic and Evolutionary Computation Conference (GECCO) in 2004, cash prizes have been awarded for human-competitive results that had been produced by some form of genetic and evolutionary computation in the previous year. 

This prize competition is based on published results. The publication must be a refereed publication in the open literature (e.g., the GECCO conference, any another reviewed conference or workshop, journal, or chapter in edited book). Submission of more than one entry by a single person or team is allowed.

The competition is open to any paper

   (1) published in the open literature between June 2, 2023 (the deadline for the previous year's competition) and May 31, 2024 (the deadline for this competition), (explicitly INCLUDING GECCO-2023 and GECCO-2024) or

   (2) that is “in press” by the deadline for this competition. "In Press" means the paper must have been unconditionally accepted for publication and be identical to that which will be published imminently without the possibility of any further changes or revision by the authors or editors. For example, a paper accepted for the current year's GECCO conference would not have been published by the deadline for the competition. However, because the paper has already been unconditionally accepted for publication (and the final camera-ready version submitted to the conference prior to the deadline for this competition), a GECCO paper is "in press."  If an entry is "in press," the entry must include a copy of the documentation establishing that the paper meets this requirement.

The paper must meet the usual standards of a scientific publication in that it must clearly describe a problem, the methods used to address the problem, the results obtained, and sufficient information about how the work was done in order to enable the work to be independently replicated.

An automatically created result is considered "human-competitive" if

it satisfies at least one of the eight criteria below.

(A) The result was patented as an invention in the past, is an improvement over a patented invention, or would qualify today as a patentable new invention.

(B) The result is equal to or better than a result that was accepted as a new scientific result at the time when it was published in a peer-reviewed scientific journal.

(C) The result is equal to or better than a result that was placed into a database or archive of results maintained by an internationally recognized panel of scientific experts.

(D) The result is publishable in its own right as a new scientific result  independent of the fact that the result was mechanically created.

(E) The result is equal to or better than the most recent human-created solution to a long-standing problem for which there has been a succession of increasingly better human-created solutions.

(F) The result is equal to or better than a result that was considered an achievement in its field at the time it was first discovered.

(G) The result solves a problem of indisputable difficulty in its field.

(H) The result holds its own or wins a regulated competition involving human contestants (in the form of either live human players or human-written computer programs).

Contestants should note that a pervasive thread in most of the above eight criteria is the notion that the result satisfy an "arms length" standard — not a yardstick based on the opinion of the author, the author's own institution (educational or corporate), or the author's own close associates. "Arms length" may be established in numerous ways. For example, if the result is a solution to "a long-standing problem for which there has been a succession of increasingly better human-created solutions," it is clear that the scientific community (not the author, the author's own institution, or the author's close associates) have vetted the significance of the problem. Similarly, a problem's significance may be established if the result replicates or improves upon a scientific result published in a peer-reviewed scientific journal, replicates or improves upon a previously patented invention, constitutes a patentable new invention, or replicates or improves a result that was considered an achievement in its field at the time it was first discovered. Similarly, a problem's significance may be established if the result holds its own or wins a regulated competition involving live human players or human-written computer programs. In each of the foregoing examples, the standard for human-competitiveness is being established external to the author, the author's own institution, or the author's close associates. It is also conceivable to rely only on criterion G ("The result solves a problem of indisputable difficulty in its field"); however, if only criterion G is claimed, there must be a clear and convincing argument that the problem's "difficulty" is indeed "indisputable."

The competition will be held as part of the annual Genetic and Evolutionary Computation (GECCO) conference.  Entries chosen to be finalists will be presented to the judges and the public in a 10-minute in-person presentation at GECCO or in a video presentation submitted to goodman at msu dot edu no later than Friday, June 28, 2024. These videos may be made available soon thereafter to the judges, but will not posted before the Humies session of the conference is held. Submission of a video constitutes agreement to posting of the video on websites of Human-Competitive.org, of GECCO, and on YouTube.  Finalists submitting videos must also submit the ACM audio/video release form allowing ACM to post it on YouTube, here. An additional form authorizing release of the video through the Linklings submission system may also be required... if so, Goodman will inform the authors and provide the form after the Finalists are selected.  The organizers believe that making these videos readily available will help to enhance the prestige of the field.  The awards and prizes will be announced at the conference.

Cash prizes of $5,000 (gold), $3,000 (silver), and bronze (either one prize of $2,000 or two prizes of $1,000) will be awarded for the best entries that satisfy one or more of the criteria for human-competitiveness.The judges may, based on submissions, reallocate the prize amounts and prize categories within the total amount available for prizes.  The awards will be divided equally among co-authors unless the authors specify a different division at the time of submission. 

Prizes are paid by the ACM by bank transfer, after the GECCO conference.   

DETAILED INSTRUCTIONS FOR ENTERING THE "HUMIES"

If you plan to make an entry into this competition, please check the web site for updated information and for possible last-minute changes immediately prior to submitting your entry. Similarly, if you are selected as a finalist, please re-check this web site frequently prior to the conference for possible last-minute changes in instructions or  schedule.

All entries are to be sent electronically to goodman at msu dot edu. All entries will be promptly acknowledged, so please make an inquiry if you do not receive a reasonably prompt acknowledgment within a few days after your submission. 

An entry must consist of one TEXT file and one or more PDF files.  In addition, if the paper is "in press" as of the deadline date for submissions, an additional document must be submitted.

If authors are making multiple entries to the competition, they should submit separate e-mails, each containing the required TEXT file and PDF file(s) supporting the entry.

The TEXT file must contain the following 11 items. Please be very careful to include ALL required information. Contestants are alerted to the fact that items 6 and 9 are especially important and will be the main basis by which entries will be judged. The papers and presentations from earlier competitions (starting in 2004) are posted at the competition web site at www.human-competitive.org. These previous entries may be informative and helpful in crafting your entry.

1. the complete title of one (or more) paper(s) published in the open literature describing the work that the author claims describes a human-competitive result;

2. the name, complete physical mailing address, e-mail address, and phone number of EACH author of EACH paper(s);

3. the name of the corresponding author (i.e., the author to whom notices will be sent concerning the competition);

4. the abstract of the paper(s);

5. a list containing one or more of the eight letters (A, B, C, D, E, F, G, or H) that correspond to the criteria (see above) that the author claims that the work satisfies;

6. a statement stating why the result satisfies the criteria that the contestant claims (see examples of statements of human-competitiveness as a guide to aid in constructing this part of the submission);

7. a full citation of the paper (that is, author names; title, publication date; name of journal, conference, or book in which article appeared; name of editors, if applicable, of the journal or edited book; publisher name; publisher city; page numbers, if applicable);

8. a statement either that "any prize money, if any, is to be divided equally among the co-authors" OR a specific percentage breakdown as to how the prize money, if any, is to be divided among the co-authors;

9. a statement stating why the authors expect that their entry would be the "best," and

10. An indication of the general type of genetic or evolutionary computation used, such as GA (genetic algorithms), GP (genetic programming), ES (evolution strategies), EP (evolutionary programming), LCS (learning classifier systems), GI (genetic improvement), GE (grammatical evolution), GEP (gene expression programming), DE (differential evolution), etc.

11. The date of publication of each paper.  If the date of publication is not on or before the deadline for submission, but instead, the paper has been unconditionally accepted for publication and is “in press” by the deadline for this competition, the entry must include a copy of the documentation establishing that the paper meets the "in press" requirement.

The PDF file(s) are to contain the paper(s). The strongly preferred method is that you send a separate PDF file for each of your paper(s) relating to your entry. Both the text file and the PDF file(s) for each entry will be permanently posted on a web page shortly after the deadline date for entries (for use by the judges, conference attendees, and anyone else who is interested) and will remain posted on the web as a permanent record of the competition. If your paper is only available on the publisher's web site and your publisher specifically requires that your published paper may appear only on your own personal page, the second choice is that you send link(s) to a separate web page on your web site containing link(s) to the PDF file(s) of the paper(s) that constitute your entry. Your publisher might require that you not make available the post-publication version of the paper, in which case you should provide a link to the final version you submitted, prior to its final formatting by the publisher.  This separate web page is to contain nothing else, so the interested parties may quickly locate your paper(s). If you use this second-choice option, you must ALSO supply a link to a permanent web site maintained by your publisher where your specific paper may be viewed or purchased (that is, not a link merely to the publisher's general home page, but a link to the specific web page containing your paper on the publisher's site). Keep in mind that the objective is to guarantee a permanent record of the entries and to make it easy for anyone to locate your material.

Generally, only one paper should be submitted. More than one paper should be submitted only if no single paper fully describes the specific result or method. Note that this competition involves specific results published in the past year, and it NOT an evaluation of the author's entire body of work over period of years. Note that this is a competition involving a result that satisfies the criteria for being "human-competitive" as defined herein.   

The judging committee will review all entries and identify a short list of finalists.  Each finalist entry must either be presented in person at GECCO in a 10-minute presentation, or if not to be presented in person, be presented in a 10-minute video submitted in advance (see below) for consideration of the judges and eventual posting for the public.  Finalists will be notified by an e-mail to the corresponding author.  Please acknowledge receipt of that message, so the judges know that you received your notice. Finalists must then either 1) submit a  10-minute video for viewing by the judging committee, showing in the Humies session, and later, posted for the public, submitted by Friday, June 28, 2024 to goodman at msu dot edu, as either the video file or a link for downloading the file (NOT a YouTube), or 2) if presenting in person, submit the overhead slides to be used for the presentation to goodman at msu dot edu no later than Friday, June 28, 2024.  The winners will be announced at the conference.  Submission of a video constitutes permission to post the video on the websites of Human-Competitive.org, GECCO, and YouTube.  All submissions will be promptly acknowledged, so please make an inquiry if you do not receive a reasonably prompt acknowledgment. These presentations will be posted on the web page for the competition. The 10-minute videos or "slides" will be viewed by the judging committee.

Any video presentations will be opened to all conference attendees at the time of the Humies session at GECCO. The video or in-person presentation should primarily focus on

1. why the result qualifies as being human-competitive and

2. why the judges should consider the entry as "best" in comparison to other entries that may also be "human-competitive."

In this 10-minute presentation to the judges, a description of the work itself is decidedly secondary. By the time of viewing of your video, the judges will be familiar with the paper(s). Thus, the focus of the presentation is on reasons why the work being presented should win a prize — not primarily an explanation or presentation of the work itself.

 

After viewing the videos and seeing any live presentations, the judges will meet to consider each finalist’s entry and decide on the winners. 

The presenting author for each entry must register for the GECCO conference.

Judges will recuse themselves if they are closely associated with a finalist (e.g., a current academic advisor, current collaborator, co-author with the finalist of related work).

See the latest, updated information at www.human-competitive.org