Python File Run Unexpected Character After Line Continuation Character
Created on 2021-11-06 13:10 by pewscorner, last changed 2022-04-11 14:59 by admin. This issue is now closed.
Messages (12) msg405860 - (view) Author: PEW's Corner (pewscorner) * Date: 2021-11-06 13:10 
Date: 2021-11-12 19:22 Date: 2021-11-14 01:06 Date: 2021-11-14 01:31 Date: 2021-11-14 01:47
When exec'ing code with an illegal character after the line continuation character '\', python 3.11.0a2 raises this strange exception: SystemError: Negative size passed to PyUnicode_New Here's an example where '#' is the illegal character: exec('1,\\#\n2') I expected the following exception (which is raised by Python 3.10.0, and also by Python 3.11.0a2 when using eval instead of exec, or when the string content is put into its own file and run as a normal script): SyntaxError: unexpected character after line continuation character msg406188 - (view) Author: Zac Hatfield-Dodds (Zac Hatfield-Dodds) * Date: 2021-11-12 05:37 It looks like this also affects Python 3.9.8, which makes me very suspicious of https://bugs.python.org/issue45494 as the probable cause. See https://github.com/Zac-HD/hypothesmith/issues/16 and https://github.com/psf/black/pull/2592#issuecomment-966745240 for reports.msg406223 - (view) Author: Matthias Bussonnier (mbussonn) * Date: 2021-11-12 17:45
Yep, this also break IPython command completions when users press tab in some places in multiline code. 88f4ec88e282bf861f0af2d237e9fe28fbc8deac is the first offending commit for me in the 3.9 branch: [3.9] bpo-45494: Fix parser crash when reporting errors involving invalid continuation characters (GH-28993) (#29071) There are two errors that this commit fixes: * The parser was not correctly computing the offset and the string source for E_LINECONT errors due to the incorrect usage of strtok(). * The parser was not correctly unwinding the call stack when a tokenizer exception happened in rules involving optionals ('?', [...]) as we always make them return valid results by using the comma operator. We need to check first if we don't have an error before continuing.. (cherry picked from commit a106343f632a99c8ebb0136fa140cf189b4a6a57) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com> NOTE: unlike the cherry-picked original, this commit points at a crazy location due to a bug in the tokenizer that required a big refactor in 3.10 to fix. We are leaving as-is for 3.9. msg406224 - (view) Author: Matthias Bussonnier (mbussonn) * Date: 2021-11-12 17:54 Note that with the following example, exec and CommandCompiler return a different error: from codeop import CommandCompiler # identical errors #exec('1,\\#\n2') SystemError #CommandCompiler()('1,\\#\n2', symbol='exec') SystemError # one is syntax the other is System. exec('a \ \n') # SyntaxError CommandCompiler()('a \ \n', symbol='exec') # SystemError msg406235 - (view) Author: Pablo Galindo Salgado (pablogsal) * Date: 2021-11-12 19:22 I will be able to take a look later in the weekend, so if someone is able to give it a go before, that would be great.msg406236 - (view) Author: Matthias Bussonnier (mbussonn) * Date: 2021-11-12 19:32
> I will be able to take a look later in the weekend Please take some time for yourself, I added you as you were the original author of the patch to make you aware. I believe most projects can workaround that for now.msg406301 - (view) Author: Pablo Galindo Salgado (pablogsal) *
Date: 2021-11-14 01:06 New changeset 25835c518aa7446f3680b62c1fb43827e0f190d9 by Pablo Galindo Salgado in branch 'main': bpo-45738: Fix computation of error location for invalid continuation (GH-29550) https://github.com/python/cpython/commit/25835c518aa7446f3680b62c1fb43827e0f190d9msg406303 - (view) Author: miss-islington (miss-islington) Date: 2021-11-14 01:30
New changeset bf26a6da7aaedb526c9eb1cb56b0e46d1c10384c by Miss Islington (bot) in branch '3.10': bpo-45738: Fix computation of error location for invalid continuation (GH-29550) https://github.com/python/cpython/commit/bf26a6da7aaedb526c9eb1cb56b0e46d1c10384cmsg406304 - (view) Author: Pablo Galindo Salgado (pablogsal) *
Date: 2021-11-14 01:31 This should be fixed by now, but please, check if everything is in order in your respective test suites and confirm here.msg406305 - (view) Author: Pablo Galindo Salgado (pablogsal) *
Date: 2021-11-14 01:47 New changeset 142fcb40b6e460fa9b4a89fe9846b1ce4176354e by Pablo Galindo Salgado in branch '3.9': bpo-45738: Fix computation of error location for invalid continuation characters in the parser (GH-29550) (GH-29552) https://github.com/python/cpython/commit/142fcb40b6e460fa9b4a89fe9846b1ce4176354emsg406322 - (view) Author: PEW's Corner (pewscorner) * Date: 2021-11-14 13:39
Seems to work. Thanks!msg406329 - (view) Author: Matthias Bussonnier (mbussonn) * Date: 2021-11-14 18:39
Works for me as well. Thanks.
History Date User Action Args 2022-04-11 14:59:52 admin set github: 89901 2021-11-14 18:39:21 mbussonn set messages: + msg406329 2021-11-14 13:59:48 pablogsal set status: open -> closed
resolution: fixed
stage: patch review -> resolved 2021-11-14 13:39:18 pewscorner set messages: + msg406322 2021-11-14 01:47:35 pablogsal set messages: + msg406305 2021-11-14 01:31:39 pablogsal set messages: + msg406304 2021-11-14 01:30:10 miss-islington set messages: + msg406303 2021-11-14 01:19:47 pablogsal set pull_requests: + pull_request27801 2021-11-14 01:07:03 miss-islington set nosy: + miss-islington
pull_requests: + pull_request27800
2021-11-14 01:06:49 pablogsal set messages: + msg406301 2021-11-14 00:03:59 pablogsal set keywords: + patch
stage: patch review
pull_requests: + pull_request27799 2021-11-12 19:32:15 mbussonn set messages: + msg406236 2021-11-12 19:30:37 pablogsal set nosy: + lys.nikolaou, BTaskaya
2021-11-12 19:22:50 pablogsal set messages: + msg406235 2021-11-12 19:08:34 aroberge set nosy: + aroberge
2021-11-12 17:54:34 mbussonn set messages: + msg406224 2021-11-12 17:45:41 mbussonn set nosy: + pablogsal
2021-11-12 17:45:20 mbussonn set nosy: + mbussonnmessages: + msg406223
2021-11-12 05:37:19 Zac Hatfield-Dodds set nosy: + Zac Hatfield-Dodds
resolution: fixed
stage: patch review -> resolved 2021-11-14 13:39:18 pewscorner set messages: + msg406322 2021-11-14 01:47:35 pablogsal set messages: + msg406305 2021-11-14 01:31:39 pablogsal set messages: + msg406304 2021-11-14 01:30:10 miss-islington set messages: + msg406303 2021-11-14 01:19:47 pablogsal set pull_requests: + pull_request27801 2021-11-14 01:07:03 miss-islington set nosy: + miss-islington
pull_requests: + pull_request27800
2021-11-14 01:06:49 pablogsal set messages: + msg406301 2021-11-14 00:03:59 pablogsal set keywords: + patch
stage: patch review
pull_requests: + pull_request27799 2021-11-12 19:32:15 mbussonn set messages: + msg406236 2021-11-12 19:30:37 pablogsal set nosy: + lys.nikolaou, BTaskaya
2021-11-12 19:22:50 pablogsal set messages: + msg406235 2021-11-12 19:08:34 aroberge set nosy: + aroberge
2021-11-12 17:54:34 mbussonn set messages: + msg406224 2021-11-12 17:45:41 mbussonn set nosy: + pablogsal
2021-11-12 17:45:20 mbussonn set nosy: + mbussonnmessages: + msg406223
2021-11-12 05:37:19 Zac Hatfield-Dodds set nosy: + Zac Hatfield-Dodds
messages: + msg406188
versions: + Python 3.9
macrossanbeent1968.blogspot.com
Source: https://bugs.python.org/issue45738
