Before leaving Halifax , it may be mentioned that the " background " school there , when compared with the K. background district in Scotland , shows differences of the same nature as those between the two complete year-groups .sx The figures are :sx THE PICTURE TESTS .sx Preliminary Trials .sx The object of the picture pages of the Group Test was to spread out the children who obtained very low marks in the Verbal Test .sx As the Otis Primary Test had been used in the survey conducted by Dr E. O. Lewis in England and Wales , two pages similar to parts of that test were designed , a Picture-Digit Substitution Test and a Pictorial Classification Test .sx These were chosen mainly on the advice of persons familiar with the Otis Test , who had found them more useful than the other portions .sx With the drawings shown in Appendix III several small trials were conducted , usually with groups of about 30 , to ascertain the best wording of the instructions for Scottish children , and to obtain some idea of the marks likely to be scored by them .sx Later a third test , Counting Cubes , was added ( see Appendix IX ) , and some small groups of children of known I.Q. were tested with all three picture pages .sx An opportunity now arose for a large-scale trial in Ilkeston ( England ) , where a whole age-group of 683 eleven-year-old children was tested , the Picture-Digit Substitution and the Pictorial Classification Tests being used , with fairly explicit instructions , not unlike those ultimately employed in Scotland .sx The results suggested that shorter instructions and timing might be sufficient .sx The next trial was the full-dress rehearsal at Halifax , where all the eleven-year-olds were tested with no exceptions other than absentees , giving a total of 660 boys and 618 girls .sx In addition , about the same number of twelve-year-olds were tested .sx This twelve-year-old group was not quite complete , having been " creamed " the previous year ; but this was largely balanced by including 93 boys and 85 girls from the secondary school .sx Lastly , the whole of one school was tested , from 8 years of age up to over 14 , adding 232 to the numbers .sx In all , over 2800 children were tested at Halifax .sx The Pictorial Classification and the Counting Cubes Tests were employed , with shorter instructions and time allowances than at Ilkeston , as follows :sx INTELLIGENCE TEST .sx Special Instructions to Supervisors ( Halifax ) .sx The Intelligence Test this year has had added to it two pages of Picture Tests .sx The booklet is an 8-page booklet , the outer page ( on which name , etc. , are to be written as usual ) being .sx page 1 , and the Picture Tests occupy pages 2 and 3 , i.e. the double page seen when the book is first opened .sx These picture pages require separate timing ( two minutes for each ) , and each supervisor must therefore see beforehand that he or she is in possession of a watch with a seconds hand .sx After the filling in of the names , etc. , at zero time the supervisor should say clearly and slowly , but without delays :sx ( Notice that these verbal instructions by the teacher are included in the two minutes .sx There should be no delay in starting to speak as soon as the children have turned over , and no " waiting for complete silence " or the like .sx ) At exactly two minutes from zero , by the seconds hand , the supervisor should say :sx ( Notice that there is no pause between the end of the first two minutes and the beginning of the second two minutes .sx ) At exactly four minutes after zero say :sx After exactly 45 minutes at the Verbal Test ( that is , 49 minutes from zero ) say Stop , and allow no more work to be done .sx So far as the Pictorial Classification Test was concerned , the Halifax results confirmed the likelihood of its success , though the instructions afterwards were again lengthened and revised and some ambiguities eliminated ( e.g. that arising from the similarity in sound of the words rose and rows) .sx It was included in the Scottish test , with the instructions and timing given in Appendix I. .sx In the Counting Cubes Test , however , not only were there , as in the other test , too many zero scores and other peculiarities , suggesting that the instructions were too short to be understood by some even of the intelligent ( that might have been obviated , as in the Pictorial Classification Test , by more explicit instructions ) , but there was also a strong sex difference , boys counting the Cubes much better than girls , and the test was therefore rejected in favour of the Picture-Digit Substitution Test .sx The Picture Tests ultimately employed in Scot-land , therefore , were Picture-Digit Substitution and Pictorial Classification .sx As the Counting Cubes data from Halifax are of some interest in themselves , and may be of service to other workers , it seems worth while to give them here ( see Appendix VIII) .sx After the Group Test had been given in Scotland , a number of reports came in indicating that the Picture-Digit Substitution Test had not worked very well .sx In part it had proved impossible to prevent children , especially dull children , from returning to this first Picture Test and spending further time on it after they found the verbal items too difficult ( in any future arrangement , if short-timed tests are bound up with block-timed tests , we shall print the former upside .sx down ) ; and in part there had been trouble over the instruction , " Now put a three under the chair just like the three under the chair in the top line .sx " Some children took this to mean a three exactly like the printed three , and wasted much time drawing very careful digits .sx As the tabulations involved an enormous amount of work , tabulation of the first Picture Test was accordingly stopped as possibly not worth while ( at any rate while more useful work could be proceeding ) , and attention was concentrated on the Pictorial Classification Test , which on the whole appeared to have worked well .sx The data for All-Scotland are given in the three tables on p. 84 .sx The equations for best-fitting lines to certain percentiles are In relation to the maximum possible score of the Pictorial Test ( 9 points ) these age allowances are of about the same size as those of the Verbal Test , namely , about 1 per cent .sx of the maximum per month ( here about 0.09 per month out of 9 , in the Verbal about 0.76 out of 76) .sx The marginal percentiles of the three large tables on p. 84 , calculated according to the conventions of Appendix VI , are .sx This shows practically no difference between the sexes .sx The discrepancy between median 7'25 and mean 6.8o is a sign of the extreme skewness of the distribution , which was intended to be only a tail , to spread the unintelligent .sx The standard deviation 2.10 is , however , even here very near to the difference between the median and the i6 percentile ( 7.25 minus ) .sx The K. " background " ( see p. 67 ) of some 500 boys and 500 girls in each of three years gave the following results with this Picture Test :sx The correlation between the verbal portion of the Group Test and the Pictorial Classification Test was first investigated by forming grids for all the June-born children in Berwickshire , Edinburgh , Shetland , Stirling-shire , Argyll and Ross , and Kincardine , together with enough boys and girls from Roxburghshire to make 500 of each ( practically the whole of Roxburghshire was needed) .sx This sample is referred to as the Bressak sample from the initials of its constituent districts .sx The correlation was found to be about 0.5. The grids appear on pp .sx 85 and 86 .sx In all the tables the correlation ratio given first is that referring to the rows and the horizontal margin .sx Later , similar grids became available for the Binet Thousand .sx They give a very similar value for the correlation of the Verbal Test with the Pictorial Classification Test .sx ANSWER PATTERNS OF THE TESTS .sx The following tables show the number of times each item was correctly answered by 500 boys and 500 girls ( in the Verbal Test ) , and by 450 boys and 450 girls ( in the Pictorial Classification Test) .sx These scripts were selected " at random " by the Marking Committees .sx A first judgment as to the random character of the selection can be made by comparing the average marks here with the median marks for All-Scotland in the following table :sx These Answer Patterns will be used to make a few changes in the serial order of the items when the test is republished .sx Helpful criticisms of certain items were also received which will lead to some minor changes .sx V .sx RELATION OF GROUP AND INDIVIDUAL .sx TESTING .sx V .sx RELATION OF GROUP AND INDIVIDUAL .sx TESTING .sx THE care taken to ensure that the Binet pupils were a fair sample of the whole population has been described above ; in the main , reliance was placed on selecting the pupils according to the census map , and in requiring that they should be born in the first week in June , though it was found necessary to relax both these restrictions very slightly .sx Despite all precautions , it was not to be expected that the sample would give a true picture of the whole population , and it has also to be borne in mind that no children in special schools were included in the Binet selection .sx The distribution of the scores of the Group Test provides a further check on the fairness of the Binet sample , and enables corrections to be made for sampling error .sx USE OF VERBAL TEST AND BINET SAMPLE TO ESTIMATE .sx BINET I.Q.'s OF THE ALL-SCOTLAND GROUP .sx The correlations of the Binet I.Q.'s with the verbal portion of the Group Test are shown in the two following grids for boys and girls .sx Calculations on these grids just as they stand give the following figures :sx If we could feel confident that these samples were , in the mathematical sense , random samples , we would conclude from these data that the boys' mean I.Q. was significantly superior to 100 , since the difference of 3 points is six times the probable error ; but that the girls' mean I.Q. was not significantly different from 100 .sx Similarly we would conclude that there was a significant difference in mean I.Q. between boys and girls , since the difference between their means can be found by calculation to be 3.9 times the probable error of that difference , corresponding to odds of about 250 to 1 against the difference being due to the chances of sampling .sx Turning to the difference between boys and girls in standard deviation of I.Q. , we would find a difference of 1.28 0.51 , which we would think less significant than that between the means , for the odds against it happening by chance sampling are only 22 to 1 .sx These conclusions would , however , be somewhat misleading .sx The samples in our case ( and almost always in experiments with human beings ) are not perfectly random :sx the children were not equally accessible as are equal-sized balls in the urn of a theoretical example in .sx probability .sx And from their verbal scores we have , what is usually lacking , the possibility of comparing our samples with the whole population which they represent , i.e. the children born in Scotland in 1921 .sx From a comparison of their verbal averages with those of the whole 1921 group , or with those of all the children born in May and June 1921 , which are given in the following table , it is clear at once that both boys and girls of .sx the Binet Group are somewhat superior samples :sx A comparison of medians instead of means leads to the same conclusion .sx A more complete comparison of the three distributions is given in the next table , and the Binet and complete May-June groups are also compared in fig. 3 ( p. 97) :sx It will be seen that the tables for both Binet boys and girls tend to have too many cases in the upper parts and too few in the lower .sx Their Binet distributions do not .sx adequately represent Scotland .sx Various statistical devices to estimate the distribution of I.Q.'s in all Scotland can be used .sx ( a ) Row Factors .sx The simplest and most readily .sx understood method is to multiply each row of the above .sx grids by the factor necessary to make its total the correct one , i.e. by the factors given in the table immediately above as the ratios of the complete unselected groups to the Binet Group .sx