Rocksolid Light

Welcome to RetroBBS

mail  files  register  newsreader  groups  login

Message-ID:  

White dwarf seeks red giant for binary relationship.

devel / comp.lang.python / Re: The GIL and PyEval_RestoreThread

SubjectAuthor
o Re: The GIL and PyEval_RestoreThreadMRAB

1
Re: The GIL and PyEval_RestoreThread

<mailman.349.1695783083.23016.python-list@python.org>

  copy mid

https://www.rocksolidbbs.com/devel/article-flat.php?id=28466&group=comp.lang.python#28466

  copy link   Newsgroups: comp.lang.python
Path: i2pn2.org!i2pn.org!news.swapon.de!fu-berlin.de!uni-berlin.de!not-for-mail
From: python@mrabarnett.plus.com (MRAB)
Newsgroups: comp.lang.python
Subject: Re: The GIL and PyEval_RestoreThread
Date: Wed, 27 Sep 2023 03:48:15 +0100
Lines: 58
Message-ID: <mailman.349.1695783083.23016.python-list@python.org>
References: <CAJANk5Ezbnadnp8Kg-yafCB3SOfrrgHwbsPBqEkzDrArWrOyEQ@mail.gmail.com>
<620973cc-7e7f-7f30-3f34-56cb05c5084b@mrabarnett.plus.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-Trace: news.uni-berlin.de pjS4eK8UVAruD6dDViZtPgtAGN+IopzsNMUHLvMhro3g==
Cancel-Lock: sha1:TQTllmA1yn99vzmWovqye7T5Xes= sha256:OOihxaioHvGvSnU6qJba7qJ+c0A+l8FCghFQxBNelfw=
Return-Path: <python@mrabarnett.plus.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
Authentication-Results: mail.python.org; dkim=pass
reason="2048-bit key; unprotected key"
header.d=plus.com header.i=@plus.com header.b=hgCluEx2;
dkim-adsp=none (unprotected policy); dkim-atps=neutral
X-Spam-Status: OK 0.004
X-Spam-Evidence: '*H*': 0.99; '*S*': 0.00; 'looks': 0.02; 'parallel':
0.05; 'thread': 0.05; 'variable': 0.05; 'else.': 0.07; 'that?':
0.07; 'from:addr:python': 0.09; 'occasionally': 0.09;
'received:192.168.1.64': 0.09; 'regex': 0.09; 'threads': 0.09;
"(i'm": 0.16; 'call,': 0.16; 'discarding': 0.16;
'from:addr:mrabarnett.plus.com': 0.16; 'from:name:mrab': 0.16;
'gil': 0.16; 'message-id:@mrabarnett.plus.com': 0.16;
'received:plus.net': 0.16; 'subject:GIL': 0.16; 'super': 0.16;
'usable': 0.16; 'void': 0.16; 'wrote:': 0.16; 'python': 0.16;
'to:addr:python-list': 0.20; 'all,': 0.20; "i've": 0.22; 'maybe':
0.22; 'code': 0.23; "wasn't": 0.26; 'local': 0.27; 'expect': 0.28;
'output': 0.28; 'wrong': 0.28; 'header:User-Agent:1': 0.30;
'seem': 0.31; 'saved': 0.31; 'think': 0.32; 'guess': 0.32;
'python-list': 0.32; 'received:192.168.1': 0.32; 'but': 0.32;
'someone': 0.34; 'release': 0.34; 'able': 0.34; 'same': 0.34;
'header:In-Reply-To:1': 0.34; 'running': 0.34; 'source': 0.36;
"it's": 0.37; 'received:192.168': 0.37; 'two': 0.39; 'quite':
0.39; 'explain': 0.40; 'happen': 0.40; 'both': 0.40; 'want': 0.40;
'should': 0.40; 'here.': 0.61; 'me.': 0.62; 'received:212': 0.62;
'pass': 0.64; 'skip:r 20': 0.64; 'well': 0.65; 'time,': 0.67;
'result,': 0.69; 'subject:The': 0.70; 'claim': 0.71;
'interesting': 0.71; 'little': 0.73; 'produces': 0.76; 'happen;':
0.84; 'locking': 0.84; 'released.': 0.84; 'restored': 0.84;
'restoring': 0.84; 'sleeping': 0.91; 'locked': 0.93
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=plus.com; s=042019;
t=1695782896; bh=sWDELBrBb1pxLUfWmkYVv0VkLIUsKItqxD8ATAyTJSo=;
h=Date:Subject:To:References:From:In-Reply-To;
b=hgCluEx2oupD/2z9LzNIwcg6xebg0MlrK542khlVHPyBRL9miYm4HLCWBs/o+ZYLH
uVqifVZ/Kx3VOTugjVu2DmGwFjyH8UEgRi2BcIm463rmdcNYYUjuE3picg34JcADtJ
pXARMpGndUMKdSrJ4zkUOHwa8xeAjGjkUBugQT3zSzqOvdMyesIwJpAfbgUckwQhcp
3Bowjgff38zd3IOBh0MU/GqowoJ45w4Lj2wJ1okF/03ZGBFNNG3c+IRJXuzWpWtnbd
C1Q+PbrtcjIHwEoPpRUmtoeFtVNqzvm2s4edP7IQ0VaW5Y1CgTS7SRfBRPxAR84+U+
uTS4aYuJv6Oug==
X-Clacks-Overhead: "GNU Terry Pratchett"
X-CM-Score: 0.00
X-CNFS-Analysis: v=2.4 cv=NfT1akP4 c=1 sm=1 tr=0 ts=651397f0
a=0nF1XD0wxitMEM03M9B4ZQ==:117 a=0nF1XD0wxitMEM03M9B4ZQ==:17
a=IkcTkHD0fZMA:10 a=9aQpAagDnl5JNM6Yv1QA:9 a=QEXdDO2ut3YA:10
X-AUTH: mrabarnett@:2500
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
Thunderbird/102.15.1
Content-Language: en-GB
In-Reply-To: <CAJANk5Ezbnadnp8Kg-yafCB3SOfrrgHwbsPBqEkzDrArWrOyEQ@mail.gmail.com>
X-CMAE-Envelope: MS4xfO4MG4fz+EyVjvW9LOJUDIfeaoD6ATEL7DQhL3CyIZsNL7YvyIW7thbAcn54e7WAQN4RXLKZyilj4+HuJ8A4FhJagis2l7jeJiqy+uQsUlVWvVKZKIoq
kftBIn0PWHjLHWmDDtKMfujf7ekO9RwAsOFbrr+NoeUh2yJHGMSivhsF6bIKHrLt6i4XB+Bjn1TZg5myEzNmo2Ftv6xjea7LLSU=
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: General discussion list for the Python programming language
<python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
<mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
<mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <620973cc-7e7f-7f30-3f34-56cb05c5084b@mrabarnett.plus.com>
X-Mailman-Original-References: <CAJANk5Ezbnadnp8Kg-yafCB3SOfrrgHwbsPBqEkzDrArWrOyEQ@mail.gmail.com>
 by: MRAB - Wed, 27 Sep 2023 02:48 UTC

On 2023-09-26 14:20, Peter Ebden via Python-list wrote:
> Hi all,
>
> I've been working on embedding Python and have an interesting case around
> locking with PyEval_RestoreThread which wasn't quite doing what I expect,
> hoping someone can explain what I should expect here.
>
> I have a little example (I'm running this in parallel from two different
> threads; I have some more C code for that but I don't think it's super
> interesting):
>
> void run_python(PyThreadState* thread) {
> LOG("Restoring thread %p...", thread);
> PyEval_RestoreThread(thread);
> LOG("Restored thread %p", thread);
> PyRun_SimpleString("import time; print('sleeping'); time.sleep(3.0)");
> LOG("Saving thread...");
> PyThreadState* saved_thread = PyEval_SaveThread();
> LOG("Saved thread %p", saved_thread);
> }
>
> This produces output like
> 11:46:48.110058893: Restoring thread 0xabc480...
> 11:46:48.110121656: Restored thread 0xabc480
> 11:46:48.110166060: Restoring thread 0xabc480...
> sleeping
> 11:46:48.110464194: Restored thread 0xabc480
> sleeping
> 11:46:51.111307541: Saving thread...
> 11:46:51.111361075: Saved thread 0xabc480
> 11:46:51.113116633: Saving thread...
> 11:46:51.113177605: Saved thread 0xabc480
>
> The thing that surprises me is that both threads seem to be able to pass
> PyEval_RestoreThread before either reaches the corresponding
> PyEval_SaveThread call, which I wasn't expecting to happen; I assumed that
> since RestoreThread acquires the GIL, that thread state would remain locked
> until it's released.
>
> I understand that the system occasionally switches threads, which I guess
> might well happen with that time.sleep() call, but I wasn't expecting the
> same thread to become usable somewhere else. Maybe I am just confusing
> things by approaching the same Python thread from multiple OS threads
> concurrently and should be managing my own locking around that?
>
Storing the result of PyEval_SaveThread in a local variable looks wrong
to me.

In the source for the regex module, I release the GIL with
PyEval_SaveThread and save its result. Then, when I want to claim the
GIL, I pass that saved value to PyEval_RestoreThread.

You seem to be releasing the GIL and discarding the result, so which
thread are you resuming when you call PyEval_RestoreThread?

It looks like you're resuming the same thread twice. As it's already
resumed the second time, no wonder it's not blocking!

devel / comp.lang.python / Re: The GIL and PyEval_RestoreThread

1
server_pubkey.txt

rocksolid light 0.9.81
clearnet tor