Reader Comments

Post a new comment on this article

PsychoPy timing: A test-script and example data

Posted by jrgray on 18 Feb 2014 at 12:44 GMT

(Posted on behalf of the PsychoPy community by Jeremy Gray, Jonathan Peirce, and Michael MacAskill.)

We are pleased that Garaizar et al. selected PsychoPy as part of their investigation of software timing for cognitive neuroscience. We are also pleased to be working together with the first author toward an empirical resolution of the discrepancies (as pertain to PsychoPy).

As noted in other comments, we believe that the apparent timing errors that were reported in the manuscript are not limitations of PsychoPy per se. To this end, we are making available a simple test script to facilitate investigation by anyone interested (https://gist.github.com/j... -- click "Download Gist", unzip the file, and run timeFrames.psyexp from within PsychoPy v1.79).

Timing visual events based on temporal intervals is known to be prone to artifacts, because those intervals often do not synchronize precisely with the hardware screen refresh interval, leading to uncertainties in the actual achieved display times. To avoid this, the test script uses a standard PsychoPy feature ("time by frames") to achieve precise control over timing, even at very short latencies (one screen refresh). The script uses another standard PsychoPy feature to estimate the actual time between screen refreshes, win.lastFrameT (an automatically-collected variable that captures the time at which the last frame was displayed).

The test script alternates between sets of black and white frames, with varying set sizes (1 to 60 frames), and then summarizes the timing statistics grouped by set size. It is suited for use as is, or in combination with an external, photodiode-based assessment. 

An example summary output from one run of the script is given below; other runs were similar when using the same computer. Of particular note, the standard deviations were small, and the means were very close to being the set size (number of frames) multiplied by the expected duration per frame (1/60th of a second). Some discrepancy from this ideal value is to be expected if the refresh rate is not exactly 1/60th of a second, or if the computer's internal clock is slightly faster or slower than the frame rate. Consumer-grade clocks are not especially accurate, and so are rarely perfectly in sync.

The following timing statistics (in seconds) were obtained over a 2 minute period when run with 32 repetitions of 6 set sizes (of 1, 3, 6, 12, 30, 60 frames per set), and strict setting = True (to check for some adverse conditions at run-time), on a MacBook Pro, Mac OS 10.8.5, retina display, 60Hz screen, NVIDIA graphics card, without internet access (wireless or wired), with bluetooth disabled, with no anti-virus software, and no USB devices attached. Assuming a 60 Hz screen refresh rate, the anticipated durations for each number of frames were: 1 frame, 0.016667 s; 3 frames, 0.05 s; 6 frames, 0.1 s; 12 frames, 0.2 s, 30 frames, 0.5 s, and 60 frames, 1 s. The times in the table were obtained from the xlsx output file and sorted by set size:

set
size color mean (s) SD (s)
1 b 0.01667 0.00006
1 w 0.01666 0.00006
3 b 0.05002 0.00005
3 w 0.05001 0.00006
6 b 0.10002 0.00006
6 w 0.10000 0.00007
12 b 0.20002 0.00006
12 w 0.20004 0.00006
30 b 0.50008 0.00006
30 w 0.50006 0.00005
60 b 1.00015 0.00005
60 w 1.00014 0.00006

On the same computer, in a longer run of 160 trials (35,840 frames, almost 10 minutes), we observed a single dropped frame. It was automatically flagged by PsychoPy with a time-stamped warning message in both the log file and the experimenter's console ("33.1681 WARNING t of last frame was 33.32ms (=1/30)"). Several such timing diagnostics are an integral part of PsychoPy, as documented online (http://www.psychopy.org/a...).

These results strongly suggest that the timing errors reported in the article were an artifact of timing by time (rather than frames), the old version PsychoPy that was used, or both.

Overall, we are confident that recent versions of PsychoPy are capable of presenting visual stimuli which are synchronized with the screen refresh with high accuracy and reliability. Such performance will require a reasonably modern (ideally non-Intel) graphics card, and sufficient memory and CPU speed. The occasional timing error or two that manages to sneak in can be automatically detected and documented. For any presentation system, computationally intensive tasks could exceed the capacity of the particular hardware; experimenters should test that the desired performance is indeed being achieved in their usage.

No competing interests declared.