RetroBBS - comp.lang.python - Using a background thread with asyncio/futures with flask

Using a background thread with asyncio/futures with flask

<mailman.123.1710955936.3452.python-list@python.org>

https://www.rocksolidbbs.com/devel/article-flat.php?id=29698&group=comp.lang.python#29698

copy link Newsgroups: comp.lang.python

Path: i2pn2.org!i2pn.org!news.swapon.de!fu-berlin.de!uni-berlin.de!not-for-mail
From: twn@thomasnyberg.com (Thomas Nyberg)
Newsgroups: comp.lang.python
Subject: Using a background thread with asyncio/futures with flask
Date: Wed, 20 Mar 2024 09:22:51 +0100
Lines: 111
Message-ID: <mailman.123.1710955936.3452.python-list@python.org>
References: <8a7231df-c30c-4fa6-9daa-546b3453b547@thomasnyberg.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Cancel-Lock: sha1:vJOaJ8L+CKG4fEdAB3v0fpTP1SA= sha256:DB0s5WVM2vKN2tfx9X1a+lF7vqtTi7BI2xGxWM+hUCE=
reason="2048-bit key; unprotected key"
header.d=thomasnyberg.com header.i=@thomasnyberg.com
header.b=VEpvtYld; dkim-adsp=pass; dkim-atps=neutral
X-Spam-Status: OK 0.003
X-Spam-Evidence: '*H*': 0.99; '*S*': 0.00; 'def': 0.04; 'debug': 0.07;
'queue': 0.07; 'serving': 0.07; 'debian': 0.09; 'freeze': 0.09;
'linux': 0.09; 'prints': 0.09; "shouldn't": 0.09; 'threads': 0.09;
'worker': 0.09; 'cheers,': 0.11; 'import': 0.15; 'deployment.':
0.16; 'gnu/linux': 0.16; 'help!': 0.16; 'instead.': 0.16;
'python3': 0.16; 'sees': 0.16; 'server,': 0.16; 'subject:thread':
0.16; 'threading': 0.16; 'warning:': 0.16; 'python': 0.16;
'server.': 0.19; 'to:addr:python-list': 0.20; 'maybe': 0.22;
"what's": 0.22; 'version': 0.23; 'install': 0.23; 'installed':
0.23; '(and': 0.25; 'flask': 0.26; 'background': 0.28; 'wrong':
0.28; 'environment': 0.29; 'header:User-Agent:1': 0.30; 'seem':
0.31; "doesn't": 0.32; 'do.': 0.32; 'subject:Using': 0.32; "i'm":
0.33; 'received:192.168.0': 0.33; 'server': 0.33; 'someone': 0.34;
'printing': 0.34; 'running': 0.34; 'trying': 0.35; 'following':
0.35; 'functions': 0.36; 'received:52': 0.37; 'received:192.168':
0.37; 'way': 0.38; 'put': 0.38; 'thanks': 0.38; 'use': 0.39;
'something': 0.40; 'hello,': 0.40; 'here.': 0.61; 'skip:o 10':
0.61; 'skip:i 20': 0.62; 'skip:o 20': 0.63; 'skip:b 10': 0.63;
'skip:t 30': 0.67; 'await': 0.69; 'obvious': 0.69; 'workers':
0.69; 'url-ip:lookup error': 0.70; 'skip:a 40': 0.70;
'production': 0.71; 'skip:f 30': 0.71; 'virtual': 0.71; 'future':
0.72; 'out,': 0.78; 'true:': 0.84; 'url:localhost': 0.84
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=thomasnyberg.com;
s=default; t=1710922973;
bh=ze6SbvCrOV/q6oxE/zozqbuCuXF03pAosXvB6qvu9WY=;
h=Date:To:From:Subject:From;
b=VEpvtYldVJNKL9IwTQP8ZviuVwYogOlUrzElPIq3no6yypHPnDy5r1H5qUpU0GY/n
qyNWj3RuQ8y4FL9Udnmnvgq1w939ypsuZzaDE7RocwwUKDr5iPa/42ds3LijnKeIAR
UKeBiVnfnNfNnAP0HkkPuf9UjW98Jy99Y8QCYXs1WddHZk7IWX1BBVuXJABTwugY59
L105x5WNcxZcj+uB8BJp8FakRrO49EVXWQD60Ou0QEaph0ZhOI26aMlKo0lzDWrGk/
BMAnDLRgzgnUV6yhP+6HsBsO8j+YqMzClrnotqudqJh5ceBCakSjQiSDdZS6QH6jIL
aFuvmwaQcOExw==
User-Agent: Mozilla Thunderbird
Content-Language: en-US
X-Mailman-Approved-At: Wed, 20 Mar 2024 13:32:14 -0400
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: General discussion list for the Python programming language
<python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
<mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
<mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <8a7231df-c30c-4fa6-9daa-546b3453b547@thomasnyberg.com>

by: Thomas Nyberg - Wed, 20 Mar 2024 08:22 UTC

Hello,

I have a simple (and not working) example of what I'm trying to do. This
is a simplified version of what I'm trying to achieve (obviously the
background workers and finalizer functions will do more later):

`app.py`

```
import asyncio
import threading
import time
from queue import Queue

from flask import Flask

in_queue = Queue()
out_queue = Queue()

def worker():
print("worker started running")
while True:
future = in_queue.get()
print(f"worker got future: {future}")
time.sleep(5)
print("worker sleeped")
out_queue.put(future)

def finalizer():
print("finalizer started running")
while True:
future = out_queue.get()
print(f"finalizer got future: {future}")
future.set_result("completed")
print("finalizer set result")

threading.Thread(target=worker, daemon=True).start()
threading.Thread(target=finalizer, daemon=True).start()

app = Flask(__name__)

@app.route("/")
async def root():
future = asyncio.get_event_loop().create_future()
in_queue.put(future)
print(f"root put future: {future}")
result = await future
return result

if __name__ == "__main__":
app.run()
```

If I start up that server, and execute `curl http://localhost:5000`, it
prints out the following in the server before hanging:

```
$ python3 app.py
worker started running
finalizer started running
* Serving Flask app 'app'
* Debug mode: off
WARNING: This is a development server. Do not use it in a production
deployment. Use a production WSGI server instead.
* Running on http://127.0.0.1:5000
Press CTRL+C to quit
root put future: <Future pending>
worker got future: <Future pending cb=[Task.task_wakeup()]>
worker sleeped
finalizer got future: <Future pending cb=[Task.task_wakeup()]>
finalizer set result
```

Judging by what's printing out, the `final result = await future`
doesn't seem to be happy here.

Maybe someone sees something obvious I'm doing wrong here? I presume I'm
mixing threads and asyncio in a way I shouldn't be.

Here's some system information (just freshly installed with pip3 install
flask[async] in a virtual environment for python version 3.11.2):

```
$ uname -a
Linux x1carbon 6.1.0-18-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.76-1
(2024-02-01) x86_64 GNU/Linux

$ python3 -V
Python 3.11.2

$ pip3 freeze
asgiref==3.7.2
blinker==1.7.0
click==8.1.7
Flask==3.0.2
itsdangerous==2.1.2
Jinja2==3.1.3
MarkupSafe==2.1.5
Werkzeug==3.0.1
```

Thanks for any help!

Cheers,
Thomas

Do not underestimate the value of print statements for debugging.

devel / comp.lang.python / Using a background thread with asyncio/futures with flask

Subject	Author
Using a background thread with asyncio/futures with flask	Thomas Nyberg