lib-python/3/test/test_tokenize.py::unmodified

== PyPy 3.12.13 (7403c30cf194, Aug 25 2026, 03:37:16) [PyPy 8.0.0-alpha0 with GCC Apple LLVM 16.0.0 (clang-1600.0.26.6)]
== macOS-15.7.7-arm64-arm-64bit little-endian
== Python build: release with_assert
== cwd: /private/tmp/buildbot-arm64/pytest/test_python_worker_81667æ
== CPU count: 8
== encodings: locale=UTF-8 FS=utf-8
== resources (3): curses,gui,network

Using random seed: 4054224020
Raised RLIMIT_NOFILE: 256 -> 1024
0:00:00 load avg: 2.78 Run 1 test sequentially in a single process
0:00:00 load avg: 2.78 [1/1] test_tokenize
test_additive (test.test_tokenize.CTokenizeTest.test_additive) ... skipped 'no c-extension _tokenize in pypy'
test_async (test.test_tokenize.CTokenizeTest.test_async) ... skipped 'no c-extension _tokenize in pypy'
test_comparison (test.test_tokenize.CTokenizeTest.test_comparison) ... skipped 'no c-extension _tokenize in pypy'
test_continuation_lines_indentation (test.test_tokenize.CTokenizeTest.test_continuation_lines_indentation) ... skipped 'no c-extension _tokenize in pypy'
test_encoding (test.test_tokenize.CTokenizeTest.test_encoding) ... skipped 'no c-extension _tokenize in pypy'
test_float (test.test_tokenize.CTokenizeTest.test_float) ... skipped 'no c-extension _tokenize in pypy'
test_function (test.test_tokenize.CTokenizeTest.test_function) ... skipped 'no c-extension _tokenize in pypy'
test_int (test.test_tokenize.CTokenizeTest.test_int) ... skipped 'no c-extension _tokenize in pypy'
test_invalid_syntax (test.test_tokenize.CTokenizeTest.test_invalid_syntax) ... skipped 'no c-extension _tokenize in pypy'
test_max_indent (test.test_tokenize.CTokenizeTest.test_max_indent) ... skipped 'no c-extension _tokenize in pypy'
test_method (test.test_tokenize.CTokenizeTest.test_method) ... skipped 'no c-extension _tokenize in pypy'
test_multiplicative (test.test_tokenize.CTokenizeTest.test_multiplicative) ... skipped 'no c-extension _tokenize in pypy'
test_selector (test.test_tokenize.CTokenizeTest.test_selector) ... skipped 'no c-extension _tokenize in pypy'
test_string (test.test_tokenize.CTokenizeTest.test_string) ... skipped 'no c-extension _tokenize in pypy'
test_tabs (test.test_tokenize.CTokenizeTest.test_tabs) ... skipped 'no c-extension _tokenize in pypy'
test_unary (test.test_tokenize.CTokenizeTest.test_unary) ... skipped 'no c-extension _tokenize in pypy'
test_unicode (test.test_tokenize.CTokenizeTest.test_unicode) ... skipped 'no c-extension _tokenize in pypy'
test_newline_at_the_end_of_buffer (test.test_tokenize.CTokenizerBufferTests.test_newline_at_the_end_of_buffer) ... 
===== script foo.py ======
===== script foo.py completed: exit code 0 ======
ok
test_additive (test.test_tokenize.GenerateTokensTest.test_additive) ... ok
test_async (test.test_tokenize.GenerateTokensTest.test_async) ... ok
test_basic (test.test_tokenize.GenerateTokensTest.test_basic) ... ok
test_closing_parenthesis_from_different_line (test.test_tokenize.GenerateTokensTest.test_closing_parenthesis_from_different_line) ... ok
test_comparison (test.test_tokenize.GenerateTokensTest.test_comparison) ... ok
test_float (test.test_tokenize.GenerateTokensTest.test_float) ... ok
test_function (test.test_tokenize.GenerateTokensTest.test_function) ... ok
test_implicit_newline (test.test_tokenize.GenerateTokensTest.test_implicit_newline) ... ok
test_int (test.test_tokenize.GenerateTokensTest.test_int) ... ok
test_invalid_readline (test.test_tokenize.GenerateTokensTest.test_invalid_readline) ... ok
test_long (test.test_tokenize.GenerateTokensTest.test_long) ... ok
test_method (test.test_tokenize.GenerateTokensTest.test_method) ... ok
test_multiline_non_ascii_fstring (test.test_tokenize.GenerateTokensTest.test_multiline_non_ascii_fstring) ... ok
test_multiline_non_ascii_fstring_with_expr (test.test_tokenize.GenerateTokensTest.test_multiline_non_ascii_fstring_with_expr) ... ok
test_multiplicative (test.test_tokenize.GenerateTokensTest.test_multiplicative) ... ok
test_newline_after_parenthesized_block_with_comment (test.test_tokenize.GenerateTokensTest.test_newline_after_parenthesized_block_with_comment) ... ok
test_non_ascii_identifiers (test.test_tokenize.GenerateTokensTest.test_non_ascii_identifiers) ... ok
test_selector (test.test_tokenize.GenerateTokensTest.test_selector) ... ok
test_shift (test.test_tokenize.GenerateTokensTest.test_shift) ... ok
test_string (test.test_tokenize.GenerateTokensTest.test_string) ... ok
test_tabs (test.test_tokenize.GenerateTokensTest.test_tabs) ... ok
test_unary (test.test_tokenize.GenerateTokensTest.test_unary) ... ok
test_underscore_literals (test.test_tokenize.GenerateTokensTest.test_underscore_literals) ... ok
test_unicode (test.test_tokenize.GenerateTokensTest.test_unicode) ... ok
test_number_followed_by_name (test.test_tokenize.InvalidPythonTests.test_number_followed_by_name) ... ok
test_number_starting_with_zero (test.test_tokenize.InvalidPythonTests.test_number_starting_with_zero) ... ok
test_bom_no_cookie (test.test_tokenize.TestDetectEncoding.test_bom_no_cookie) ... ok
test_cookie_first_line_no_bom (test.test_tokenize.TestDetectEncoding.test_cookie_first_line_no_bom) ... ok
test_cookie_second_line_commented_first_line (test.test_tokenize.TestDetectEncoding.test_cookie_second_line_commented_first_line) ... ok
test_cookie_second_line_empty_first_line (test.test_tokenize.TestDetectEncoding.test_cookie_second_line_empty_first_line) ... ok
test_cookie_second_line_no_bom (test.test_tokenize.TestDetectEncoding.test_cookie_second_line_no_bom) ... ok
test_cookie_second_line_noncommented_first_line (test.test_tokenize.TestDetectEncoding.test_cookie_second_line_noncommented_first_line) ... ok
test_false_encoding (test.test_tokenize.TestDetectEncoding.test_false_encoding) ... ok
test_filename_in_exception (test.test_tokenize.TestDetectEncoding.test_filename_in_exception) ... ok
test_latin1_normalization (test.test_tokenize.TestDetectEncoding.test_latin1_normalization) ... ok
test_matched_bom_and_cookie_first_line (test.test_tokenize.TestDetectEncoding.test_matched_bom_and_cookie_first_line) ... ok
test_matched_bom_and_cookie_second_line (test.test_tokenize.TestDetectEncoding.test_matched_bom_and_cookie_second_line) ... ok
test_mismatched_bom_and_cookie_first_line_raises_syntaxerror (test.test_tokenize.TestDetectEncoding.test_mismatched_bom_and_cookie_first_line_raises_syntaxerror) ... ok
test_mismatched_bom_and_cookie_second_line_raises_syntaxerror (test.test_tokenize.TestDetectEncoding.test_mismatched_bom_and_cookie_second_line_raises_syntaxerror) ... ok
test_no_bom_no_encoding_cookie (test.test_tokenize.TestDetectEncoding.test_no_bom_no_encoding_cookie) ... ok
test_open (test.test_tokenize.TestDetectEncoding.test_open) ... ok
test_open_error (test.test_tokenize.TestDetectEncoding.test_open_error) ... ok
test_short_files (test.test_tokenize.TestDetectEncoding.test_short_files) ... ok
test_syntaxerror_latin1 (test.test_tokenize.TestDetectEncoding.test_syntaxerror_latin1) ... ok
test_utf8_normalization (test.test_tokenize.TestDetectEncoding.test_utf8_normalization) ... ok
test_decistmt (test.test_tokenize.TestMisc.test_decistmt) ... ok
test_backslash_continuation (test.test_tokenize.TestRoundtrip.test_backslash_continuation) ... ok
test_continuation (test.test_tokenize.TestRoundtrip.test_continuation) ... ok
test_indentation_semantics_retained (test.test_tokenize.TestRoundtrip.test_indentation_semantics_retained)
Ensure that although whitespace might be mutated in a roundtrip, ... ok
test_random_files (test.test_tokenize.TestRoundtrip.test_random_files) ... 
  test_random_files (test.test_tokenize.TestRoundtrip.test_random_files) (file='/Users/matti/build-worker-arm64/pypy-c-jit-macos-arm64/build/lib-python/3/test/test_grammar.py') ... FAIL
test_roundtrip (test.test_tokenize.TestRoundtrip.test_roundtrip) ... ok
test_string_concatenation (test.test_tokenize.TestRoundtrip.test_string_concatenation) ... ok
test_comment_at_the_end_of_the_source_without_newline (test.test_tokenize.TestTokenize.test_comment_at_the_end_of_the_source_without_newline) ... ok
test_exact_type (test.test_tokenize.TestTokenize.test_exact_type) ... ok
test_invalid_character_in_fstring_middle (test.test_tokenize.TestTokenize.test_invalid_character_in_fstring_middle) ... ok
test_newline_and_space_at_the_end_of_the_source_without_newline (test.test_tokenize.TestTokenize.test_newline_and_space_at_the_end_of_the_source_without_newline) ... ok
test_oneline_defs (test.test_tokenize.TestTokenize.test_oneline_defs) ... ok
test_pathological_trailing_whitespace (test.test_tokenize.TestTokenize.test_pathological_trailing_whitespace) ... ok
test_tokenize (test.test_tokenize.TestTokenize.test_tokenize) ... ok
test_bad_coding_cookie (test.test_tokenize.TestTokenizerAdheresToPep0263.test_bad_coding_cookie) ... ok
test_latin1_coding_cookie_and_utf8_bom (test.test_tokenize.TestTokenizerAdheresToPep0263.test_latin1_coding_cookie_and_utf8_bom)
As per PEP 0263, if a file starts with a utf-8 BOM signature, the only ... ok
test_no_coding_cookie_and_utf8_bom (test.test_tokenize.TestTokenizerAdheresToPep0263.test_no_coding_cookie_and_utf8_bom) ... ok
test_utf8_coding_cookie_and_no_utf8_bom (test.test_tokenize.TestTokenizerAdheresToPep0263.test_utf8_coding_cookie_and_no_utf8_bom) ... ok
test_utf8_coding_cookie_and_utf8_bom (test.test_tokenize.TestTokenizerAdheresToPep0263.test_utf8_coding_cookie_and_utf8_bom) ... ok
test__tokenize_decodes_with_specified_encoding (test.test_tokenize.Test_Tokenize.test__tokenize_decodes_with_specified_encoding) ... ok
test_additive (test.test_tokenize.TokenizeTest.test_additive) ... ok
test_async (test.test_tokenize.TokenizeTest.test_async) ... ok
test_basic (test.test_tokenize.TokenizeTest.test_basic) ... ok
test_closing_parenthesis_from_different_line (test.test_tokenize.TokenizeTest.test_closing_parenthesis_from_different_line) ... ok
test_comparison (test.test_tokenize.TokenizeTest.test_comparison) ... ok
test_float (test.test_tokenize.TokenizeTest.test_float) ... ok
test_function (test.test_tokenize.TokenizeTest.test_function) ... ok
test_implicit_newline (test.test_tokenize.TokenizeTest.test_implicit_newline) ... ok
test_int (test.test_tokenize.TokenizeTest.test_int) ... ok
test_invalid_readline (test.test_tokenize.TokenizeTest.test_invalid_readline) ... ok
test_long (test.test_tokenize.TokenizeTest.test_long) ... ok
test_method (test.test_tokenize.TokenizeTest.test_method) ... ok
test_multiline_non_ascii_fstring (test.test_tokenize.TokenizeTest.test_multiline_non_ascii_fstring) ... ok
test_multiline_non_ascii_fstring_with_expr (test.test_tokenize.TokenizeTest.test_multiline_non_ascii_fstring_with_expr) ... ok
test_multiplicative (test.test_tokenize.TokenizeTest.test_multiplicative) ... ok
test_newline_after_parenthesized_block_with_comment (test.test_tokenize.TokenizeTest.test_newline_after_parenthesized_block_with_comment) ... ok
test_non_ascii_identifiers (test.test_tokenize.TokenizeTest.test_non_ascii_identifiers) ... ok
test_selector (test.test_tokenize.TokenizeTest.test_selector) ... ok
test_shift (test.test_tokenize.TokenizeTest.test_shift) ... ok
test_string (test.test_tokenize.TokenizeTest.test_string) ... ok
test_tabs (test.test_tokenize.TokenizeTest.test_tabs) ... ok
test_unary (test.test_tokenize.TokenizeTest.test_unary) ... ok
test_underscore_literals (test.test_tokenize.TokenizeTest.test_underscore_literals) ... ok
test_unicode (test.test_tokenize.TokenizeTest.test_unicode) ... ok
test_backslash_continuation (test.test_tokenize.UntokenizeTest.test_backslash_continuation) ... ok
test_bad_input_order (test.test_tokenize.UntokenizeTest.test_bad_input_order) ... ok
test_iter_compat (test.test_tokenize.UntokenizeTest.test_iter_compat) ... ok

======================================================================
FAIL: test_random_files (test.test_tokenize.TestRoundtrip.test_random_files) (file='/Users/matti/build-worker-arm64/pypy-c-jit-macos-arm64/build/lib-python/3/test/test_grammar.py')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/matti/build-worker-arm64/pypy-c-jit-macos-arm64/build/lib-python/3/test/test_tokenize.py", line 2030, in test_random_files
    self.check_roundtrip(f)
         ^^^^^^^^^^^^^^^
  File "/Users/matti/build-worker-arm64/pypy-c-jit-macos-arm64/build/lib-python/3/test/test_tokenize.py", line 1867, in check_roundtrip
    self.assertEqual(tokens2_from2, tokens2)
         ^^^^^^^^^^^
AssertionError: Lists differ: [(67,[192002 chars]64, ' '), (64, '"'), (59, '{'), (1, 'test1'), [674 chars] '')] != [(67,[192002 chars]64, '"'), (59, '{'), (1, 'test1'), (59, '('), [663 chars] '')]

First differing element 14623:
(64, ' ')
(64, '"')

First list contains 1 additional elements.
First extra element 14678:
(0, '')

Diff is 237039 characters long. Set self.maxDiff to None to see it.

----------------------------------------------------------------------
Ran 110 tests in 0.789s

FAILED (failures=1, skipped=17)
0:00:00 load avg: 2.78 [1/1/1] test_tokenize failed (1 failure)

== Tests result: FAILURE ==

1 test failed:
    test_tokenize

Total duration: 961 ms
Total tests: run=110 failures=1 skipped=17
Total test files: run=1/1 failed=1
Result: FAILURE
test test_tokenize failed

builder: pypy-c-jit-macos-arm64 build #1390+

test: lib-python/3/test/test_tokenize.py::unmodified