The Open System of Judging

The following is an updating on the original piece Shep Paine wrote about the system, revised by Bill Horan for World Expo Chicago 2017, further edited by MMSI Chief Judge Jim DeRogatis in 2022, and available for any and all clubs that want some guidance. (And keep in mind, it is a constant work in progress, always open to further clarification and improvement!)

INTRODUCTION

The basic premise of the Open System of Judging as developed by Shep Paine and other top modelers from around the world in the 1980s is to recognize the good modelers and encourage the promising ones at a “juried exhibition” (not a contest). No deserving work should go home without having been assessed and possibly rewarded. Anything done to further this goal is a good idea, and anything that impedes it is a bad one.

The system is intended to counter the weaknesses we've seen in other shows, particularly those which use category systems, with first-, second-, and third-place awards in each category. The problem is that the competition is always stiffer in some classes than others, and the fourth-place loser in a crowded category is often much better than the first-place winner in another. Thus, a first-rate piece goes home empty-handed, while a mediocre work gets recognition it does not necessarily deserve. And the majority of modelers get no sense whatsoever where their work fell in terms of all the pieces on display and the state of the hobby in general.

The Open System reduces the categories to a minimum and eliminates the fixed number of awards, allowing the level of work on display to determine the number and type of awards given at the show.

Perhaps the most interesting feature of this system is that it is not a competitive one. Because the number of awards is expandable, no piece wins an award at the expense of another. Each exhibitor's work is evaluated on its own, and he or she receives the level of award the judges feel the work merits, without regard to the awards given to other pieces in the hall. Painters can thus judge the progress of their work from show to show, without worrying about what other modelers may bring.

The standard against which the exhibits are judged is set by winners in previous years as well as the overall standard of the current show. An experienced judge can look at a piece and say, "That is as good as the pieces that won Silver Medals last year," and score it accordingly. Judges have, of course, different opinions, which is why we use a team of three of them.

A disadvantage of this system is that one really has to see it in operation to appreciate some of its subtler aspects. Nonetheless, we hope this detailed explanation of its workings will help those interested in learning how it works. Meanwhile, this document has been compiled with three goals in mind: First, to provide a uniform set of criteria for judging figures, dioramas, and ordnance at shows such as those presented by the Military Miniatures Society of Illinois, the Miniature Figure Collectors of America, the Atlanta Military Figures Society, the World Expo, and others; Second, to form a sound basic text for the training of judges in the future, and Third, to give exhibitors specific information about what the judges will be looking for.

These are not rules; they are a set of artistic criteria, by which judges can compare models uniformly and fairly. As such, they can effectively be applied to virtually any exhibition, regardless of the rules under which it may be run. The key element in fair judging is that the judges work to the same set of criteria in assessing the entries. That said, the criteria presented here offer few absolutes. Qualifiers like "sometimes," "often," and "can be" are frequent. This is both appropriate and inevitable. Judging is ultimately a matter of personal opinion, and while that opinion can be guided, it cannot be dictated. In the final analysis, these criteria should therefore be viewed more as guidelines than a checklist.

We propose the following five criteria, not necessarily listed in order of importance or consideration.

• DEGREE OF DIFFICULTY

• CREATIVITY

• WORKMANSHIP

• PAINTING SKILL

• PRESENTATION & OVERALL EFFECT

We will discuss each of these criteria in turn, pointing out as specifically as we can the manner in which each relates to the exhibits in these key categories:

• HISTORICAL PAINTERS

• HISTORICAL OPEN

• FANTASY PAINTERS

• FANTASY OPEN

• ORDNANCE PAINTERS

• ORDNANCE OPEN

In the Painters Division, judging is based entirely upon painting skill. Although scratchbuilt and converted figures are not excluded, they are judged as if they were stock commercial castings. The Open Division, on the other hand, is intended primarily for scratchbuilt and converted figures, vignettes, and dioramas; exhibits are judged on equal measures of imagination and skill. The Ordnance division is for models without figures. Figures may be included, but they will not count in the judging.

DEGREE OF DIFFICULTY

For stock commercial figures, consideration starts with the casting itself. A beautifully detailed and well-proportioned figure is plainly easier to cope with than one which must be extensively re-worked to bring it up to standard. But the main area of attention for stock figures is clearly the painting. Although intricately painted detail is certainly impressive, judges should not ignore the fact that the subtle shading of long flowing robes with no detail at all can be equally challenging. Also worth bearing in mind is that dark colors tend to be easier to shade than light ones. Moreover, experience has shown that larger figures are harder to paint than smaller ones. Much of the success of a figure depends on the face, and the amount of character and detail instilled by the manufacturer can make this job much easier. Mounted figures present the additional problem of the horse, which must be delicately shaded; dapples and greys are much more difficult in this regard. Flats are often more difficult, since a genuine feeling of roundness must be achieved with the paintbrush alone.

Figures in different scales raise another question. A larger figure is generally more challenging to paint, convert, or sculpt than a small one; a 90mm figure is harder than a 54mm.

For Conversions, the amount of conversion work attempted becomes the key factor. A minor conversion would consist only of a small change in pose or subject, one in which the manufacturer's original design is plainly evident. A major conversion generally involves extensive reworking of the pose or uniform, but one in which the original design can still be discerned, albeit with some difficulty. A complete conversion leaves practically nothing of the original design except the face and the hands, and these too are sometimes altered. The challenge of a chosen subject is also an important consideration. Intricate detail, certain difficult textures (smooth, crisply defined armor, for example), and facial expressions (subtle smiles and open mouths) all require more effort. Although dramatic action poses are exciting and difficult to carry off, judges should be aware that subtle execution of relaxed poses can often prove equally frustrating. Scratchbuilt projects offer many of the same difficulties as conversions, with a few others thrown in for good measure. Pose selection and execution become more critical, with figure proportions harder to control. Facial features and expression are significant obstacles. Judges should weigh the fact that gnarled, expressive faces are sometimes easier to accomplish than the subtlety of smoother faces in repose, and a beautiful woman can be the greatest challenge of all. Intricate detail and long flowing robes are each difficult in their own way. Horses present problems of anatomy and musculature that require as much skill as a human figure.

Many of the considerations mentioned so far carry over into the fields of dioramas and vignettes, since these often involve the use of converted or scratchbuilt figures. But scenes of this sort present other problems as well, ones of design, effective storytelling, carrying an idea across to the viewer, creating a desired mood, composing a scene with a smooth flow of action, and effective use of the available space. Buildings, settings, and accessories must often be converted or built from scratch, while boxed dioramas involve the additional problems of lighting and electricity. The sheer number of figures, while sometimes a minor factor, should not in itself carry much weight. Only when the difficulty of a project has been assessed can the judges proceed to evaluate the work that has been done.

CREATIVITY

The creativity a modeler shows in presenting his subject can take a surprising variety of forms. Although we have all had occasions to see certain models and exclaim "why didn't I think of that," creativity as intended here should not be confused with originality. All too often one finds that what one took to be original is in fact based upon an obscure illustration, or even another model from a different period.

Although imagination in choice of subject should not be ignored, it can also be evident in other areas, such as making ingenious use of a stock figure for a conversion, choice of an unusual setting for a fairly commonplace idea, creating a scene by clever combination of stock figures, or the inspired adaptation of unusual materials to achieve a certain effect. Creativity rarely plays a role at all with stock commercial figures, since the modeler simply paints it according to the instructions provided.

For conversions and scratchbuilt figures, imagination plays an important part, primarily in the selection of subject and pose (although with conversions, the creative adaptation of stock figures, commercial parts, and everyday "found" items certainly contribute as well).

With dioramas and vignettes, creativity manifests itself in the choice of subject, setting, and lighting (if any), in addition to the factors involved with the figures. Judges should also keep a sharp eye out for telling little details, although they should avoid rewarding cleverness that exists only for its own sake, unconnected to the scene. Creativity and the effectiveness of storytelling are two of the most important judging criteria with dioramas and vignettes. After all, it is the continuing inventiveness and ingenuity of modelers that pushes the state of the hobby to ever greater heights.

WORKMANSHIP

Under the heading of workmanship we bring together the technical execution of all the non-painting efforts required for the project. For stock commercial figures, this involves little more than ensuring proper removal of molding seams and evaluating the success of any sharpening of detail that may have been attempted. Conversions and scratchbuilts are a different matter, since even the best paint job cannot make up for a clumsily converted or poorly sculpted figure. What we are looking for here is performance: how naturally the figure is posed, how well the clothing folds are handled, how crisply the piece is detailed. For both conversions and scratchbuilts, correct anatomy is paramount: an ill-proportioned or unnaturally posed figure should be penalized, however beautifully it may be detailed and painted. With conversions, particular attention should be paid to the specific areas where the conversion work has been done, ensuring first that modified joints are smoothly covered, with any disturbed clothing folds carefully restored, and second that any new detail work is neat and precisely rendered.

Workmanship assumes an even greater importance in dioramas and vignettes, where, in addition to the figures themselves, it encompasses everything from the construction of settings and groundwork to the effectiveness of the lighting. Any buildings, vehicles, and accessories involved in the scene should be well constructed and properly detailed, with effective use of materials. In boxed dioramas, the sight lines should be contrived to conceal all that is not meant to be seen, and the lighting arranged so that it achieves the desired effect while still illuminating the scene. If forced perspective is employed, it should work effectively, without distracting the viewer.

PAINTING SKILL

Painting skill is undoubtedly the single most important criteria, since a failure in this area can seriously undermine the effect of even the most beautifully sculpted figure or the most imaginatively contrived scene. Painting skill is practically the only criteria when it comes to stock commercial figures. The neatness of the detail and the subtlety of the shading are important considerations, but in most cases it is the face that really measures the success or failure of the piece. The eyes should be accurately located, level with each other, and lacking any suggestion of a "pop-eyed" look. The facial planes should be strongly rendered, yet subtly shaded. Above all, the face should have character and life suggesting a real person, not a waxen, dead image.

Care should be taken that the variety of textures are accurately portrayed, from the coarse wool of a World War I tunic to the soft sheen of well-oiled leather. Soldiers on campaign should have the weathered, worn appearance commensurate with their situation; spotless uniforms and spit-shined leather belong on the parade ground. Horses should be subtly shaded. yet still show a strong, confident use of color. The painting should accurately reflect the anatomy and physical characteristics of the animal, such as the gray areas around the eyes and mouth, the proper coloring of the hooves, the sparse hair areas, reproductive organs, eye color, and markings. For proper appearance, dappling should be subtle in appearance and varied in size.

It is important to separate style from competent technique, and judges should be aware of their own prejudices in this regard. Every judge has encountered painting styles they didn't particularly care for, but it is important to keep in mind that this is very much a matter of personal taste; just because one doesn't fancy a particular style does not make it wrong. The proper approach is to question whether, within the given style, the painting is skillfully done.

Conversions and Scratchbuilts obviously call for the same painting criteria as stock commercial figures. Dioramas and vignettes, on the other hand, frequently involve structures, vehicles, artillery pieces, or other accessories, and due attention must be paid to these. The first concern is that the same loving care and effort be devoted to these as to the figures. Moreover, it is important that there be a unity of style between the figures and the other elements of the scene; in other words, if the figures are dramatically shaded, the vehicles and buildings should be, too. The feeling should be one of a scene rendered with the same brush, rather than a collection of disconnected elements painted separately. Both buildings and vehicles should be neatly detailed, subtly shaded, and weathered to a degree appropriate to the circumstances.

PRESENTATION AND OVERALL EFFECT

The first part of this criterion is fairly straightforward, concerning itself with the base, groundwork, and any other elements involved in "presenting" the piece. The base and groundwork are not trivial considerations. Anyone who has ever judged can recall examples of nicely done figures and scenes where the amateurish appearance of the base and groundwork badly undermined the positive impression made by the model itself.

Although the style of the base used can vary widely according to the modeler's taste, whatever is chosen should be neatly and tastefully finished; unvarnished plywood and crudely cut styrofoam are offensive to the eye. The groundwork and vegetation should be realistic in effect, appropriate to the geographic setting, and painted in a style commensurate with the figures. Additional credit should be given for properly creative accessories or battlefield debris, but these should be suited to the subject and not just gratuitous detail added to fill space or curry favor with the judges. For boxed dioramas, presentation is limited to the box itself and the framing of the scene, which should be complementary to the subject, neatly done, and unobtrusive.

The second half of this criteria, overall effect," is really the more important of the two, yet harder to define. What it really amounts to is a recognition of the intangible aspect of modeling, the effectiveness of the storytelling, and that "feeling of life" which cannot be traced to any particular element of its construction, but which is clearly evident when the piece is viewed as a whole. As such, its precise nature can only be vague and slightly mysterious; ultimately, it must remain very much a matter of the personal reaction of the judges.

A good overall effect can sometimes raise a marginally less skillful effort over its more finely executed but sterile counterpart, and is often the ultimate determining factor in close decisions when all other tie-breakers fail. Achieving this "feeling of life" is really the ultimate goal of the hobby and, for this reason if no other, it should never be forgotten or overlooked by the judges.

HISTORICAL ACCURACY

Readers will notice the complete absence of "historical accuracy" as a criteria. This is no oversight. The problem is that with the multitude of subjects seen at shows today, it is simply impossible to judge all models with equal authority, even within a narrow historical period. An entry on a familiar subject is likely to be penalized for even the slightest error, while major errors in a more obscure subject escape totally unnoticed. Even acknowledged experts in a given field (and there are few enough of these) cannot possibly carry enough information in their heads to judge all entries in that field fairly.

Moreover, while minor errors can always be found, it should also be recognized that in a day when kits are provided with coloring instructions, the lack of historical accuracy is not a major problem in figure exhibitions. Many modelers are themselves amateur historians, who do original research of their own. Surely it is better to let a few historical culprits go free than to unjustly penalize an enterprising researcher for information the judge could not have been aware of. Ultimately, the best advice for judges is this: If you see an obvious and blatant error, it cannot help but shade your judgment, but if there is any question, give the competitor the benefit of that doubt. They have devoted more time to the model than you have, and just may be right.

We acknowledge that there will be those who disagree with this view. If they wish to add historical accuracy as a criteria, they are certainly free to do so. We would suggest, however, that they a) publish a list of criteria similar to those presented here, and b) require their exhibitors to submit a brief list of their references used in preparing their exhibit. If the error noted was the fault of the reference, and not the modeler, they should not be penalized for it.

BALANCE OF THE CRITERIA

Obviously, certain criteria are more important in judging some types of exhibits than others. The table below indicates the approximate weight that should be given to the various criteria when judging different types of entries.This percentage breakdown can be used as an actual point/scoring system, or less formally as a general guide for the judges in reaching their decisions. Another workable option would be to combine the two, employing the general guide for most decisions, while failing back on the point/scoring system in cases where the judges cannot agree. This would ensure that close decisions are made faithfully following the criteria, yet free the judges of the burden of having to score every entry.

It is important to note that these percentages hold true only when comparing models within each group, and should never be used to compare different types of exhibits. The balance of the criteria have been presented in percentage form so that societies wishing to set up their own point system have a firm but flexible basis on which to do so.

UNDERSTANDING THE CATEGORIES

(Which should I enter?)

BASIC LEVEL

Intended for painters and modelers of modest ability or those who are just starting out. Qualifying exhibits receive certificates of merit.

JUNIORS LEVEL

For young modelers up to age 16, although if they are confident in their abilities, they are free to enter at the advance level. Qualifying exhibits are eligible for a juniors medal or certificates of merit.

ADVANCED LEVEL (SIX CATEGORIES)

HISTORICAL PAINTERS

Historical or reality-based subjects from stock kits or with very minor conversions. Entries judged primarily on the quality of the painting.

HISTORICAL OPEN

Historical or reality-based subjects with heavily converted or scratchbuilt figures, as well as dioramas and vignettes. Judged on sculpting, painting, overall presentation, and storytelling.

FANTASY PAINTERS

Fantasy subjects from stock kits or with very minor conversions. Entries judged primarily on the quality of the painting.

FANTASY OPEN

Fantasy subjects incorporating heavily converted or scratchbuilt figures, as well as dioramas and vignettes. Judged on sculpting, painting, overall presentation, and storytelling.

ORDNANCE PAINTERS

ORDNANCE OPEN

Armor, aircraft, wheeled vehicles, robots, and fantasy vehicles that may or may not include figures and scenery but primarily focus on the vehicles and mechanical equipment. Specifically for scratchbuilt subjects or kits which include extensive extra detailing or conversion work. Judging centers on the modeling and painting skills of the vehicle. Artists who place equal emphasis on all elements of a diorama or vignette may consider entering Historical or Fantasy Open if the vehicle is not their primary/only focus in the piece.HISTORICAL WARGAME

For painted wargame figures. Qualifying exhibits are eligible for certificates of merit or a wargame medal.

TOY SOLDIERS

For collectors who want to display toy soldiers (including those they’ve painted or simply collected). Qualifying exhibits are eligible for certificates of merit or a toy soldiers medal.