RetroBBS - comp.lang.python - Re: Using a background thread with asyncio/futures with flask

Re: Using a background thread with asyncio/futures with flask

<mailman.8.1711102188.3468.python-list@python.org>

https://www.rocksolidbbs.com/devel/article-flat.php?id=29716&group=comp.lang.python#29716

copy link Newsgroups: comp.lang.python

Path: i2pn2.org!i2pn.org!news.chmurka.net!weretis.net!feeder8.news.weretis.net!fu-berlin.de!uni-berlin.de!not-for-mail
From: frank@chagford.com (Frank Millman)
Newsgroups: comp.lang.python
Subject: Re: Using a background thread with asyncio/futures with flask
Date: Fri, 22 Mar 2024 12:09:10 +0200
Lines: 125
Message-ID: <mailman.8.1711102188.3468.python-list@python.org>
References: <8a7231df-c30c-4fa6-9daa-546b3453b547@thomasnyberg.com>
<7af73918-8209-4393-8aa2-140940567456@chagford.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Trace: news.uni-berlin.de 9L9Y6GXubloV+7GnZVZGsg4ao4WN/uV29fBngxyL7iIQ==
Cancel-Lock: sha1:+Kpx67LCNcPyLBOC5SNP/XFiZvo= sha256:HXj+MI5cfTWbTC2+Qy60aO3FYWb/1NwH/FebbVDs81c=
Return-Path: <frank@chagford.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
Authentication-Results: mail.python.org; dkim=pass
reason="2048-bit key; unprotected key"
header.d=chagford.com header.i=@chagford.com header.b=oJHTwUEl;
dkim-adsp=pass; dkim-atps=neutral
X-Spam-Status: OK 0.000
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'def': 0.04; 'debug': 0.07;
'queue': 0.07; 'serving': 0.07; 'debian': 0.09; 'freeze': 0.09;
'linux': 0.09; 'prints': 0.09; "shouldn't": 0.09; 'threads': 0.09;
'worker': 0.09; 'cheers,': 0.11; 'import': 0.15; 'app,': 0.16;
'deployment.': 0.16; 'expert.': 0.16; 'from:addr:chagford.com':
0.16; 'from:addr:frank': 0.16; 'from:name:frank millman': 0.16;
'gnu/linux': 0.16; 'help!': 0.16; 'instead.': 0.16; 'message-
id:@chagford.com': 0.16; 'millman': 0.16; 'python3': 0.16;
'received:196.35': 0.16; 'received:196.35.198': 0.16;
'received:197.90': 0.16; 'received:197.90.32': 0.16;
'received:197.90.32.26': 0.16; 'received:synaq.com': 0.16; 'sees':
0.16; 'server,': 0.16; 'subject:thread': 0.16; 'threading': 0.16;
'warning:': 0.16; 'wrote:': 0.16; 'python': 0.16; 'server.': 0.19;
'to:addr:python-list': 0.20; 'maybe': 0.22; "what's": 0.22;
'version': 0.23; 'install': 0.23; 'installed': 0.23; '(and': 0.25;
'flask': 0.26; 'background': 0.28; 'wrong': 0.28; 'environment':
0.29; 'header:User-Agent:1': 0.30; 'seem': 0.31; 'am,': 0.31;
"doesn't": 0.32; 'do.': 0.32; 'python-list': 0.32;
'subject:Using': 0.32; "i'm": 0.33; 'server': 0.33; 'someone':
0.34; 'printing': 0.34; 'header:In-Reply-To:1': 0.34; 'running':
0.34; 'trying': 0.35; 'following': 0.35; 'functions': 0.36; 'way':
0.38; 'put': 0.38; 'thanks': 0.38; 'use': 0.39; 'something': 0.40;
'hello,': 0.40; 'here.': 0.61; 'skip:o 10': 0.61; 'skip:\xc2 10':
0.62; 'skip:i 20': 0.62; 'skip:o 20': 0.63; 'skip:b 10': 0.63;
'similar': 0.65; 'skip:t 30': 0.67; 'await': 0.69; 'obvious':
0.69; 'received:196': 0.69; 'workers': 0.69; 'url-ip:lookup
error': 0.70; 'skip:a 40': 0.70; 'production': 0.71; 'skip:f 30':
0.71; 'virtual': 0.71; 'future': 0.72; '8bit%:100': 0.76; 'out,':
0.78; 'true:': 0.84; 'url:localhost': 0.84
ARC-Seal: i=1; cv=none; a=rsa-sha256; d=synaq.com; s=securemail; t=1711102186;
b=mkgNPG5pl/9qVENeyLGzmHXr2RjuSDjD00xBKi2ohHGB9h4+VDu2EkOIs1xl2q89fQz68O04Nu
Gw3gDzyvSsEcLL/GPni1ycUZ1wJ0N1s8yicvfafMwmtMlnGXbak+bMGZypAkuK7TAcqGvkZ4Nf
6ZJxjdRJ+aMeX9VUcx0r7aCpNJMu6k+hpD2w1aTKfkPiAFk9Eo7HVAFp8AnKgfde7qW/Sv1uuI
4UjiONCO0jvhxjdNdI4bKVFn4IMOZfKxZwxkq2ZTP0yUUimO9r4i12DDAa3ss9vBirJD7EvM2z
ahnZ+7kYLNR4fS2BCOhiCaYiVgTBLeL6KS8ir56KGpAn8Q==;
ARC-Authentication-Results: i=1; synaq.com;
iprev=fail smtp.remote-ip=197.90.32.26;
auth=pass (PLAIN) smtp.auth=frank@chagford.com;
arc=none
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed; d=synaq.com; s=securemail;
t=1711102186;
bh=Hew85YGPVXBUo+0Cf+JF9wYrsSbgZnn48AnOitCsbA0=;
h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:References:To:
Subject:MIME-Version:Date:Message-ID:DKIM-Signature;
b=R2XL0zMTZ3m1ddZO/w90S0EzmCMv50aUDjCEsolgS2KJmRPjbD9SBuPDuxrvMNDH9PKYn0VDis
cA3RCi2VmhHG+DW6hQulFaG4NjjhjiMJLFcgpLJYC3hacfQAZoU68vNYLoPUW+2XYlMuKEkBC1
8+BVfI7uHpS24Wh0fmSXjIuVglhWhqTqIyvXU3tnH63WF4cfs8LKfbPpifRbfqBwIyi0uNKYZP
1+twbvD5oa2hUbdBp1RVeBiprbgbKzbsApfqWtizGjhfWHRZ6YdP9+jvIIQRUXGtdcny5/dTm4
hgrVRS2XLxfHLMsaS9u1/r7vWakLQNUJwMqDdiN2uQCveQ==;
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
d=chagford.com; s=securemail; h=In-Reply-To:From:To:Date:Message-ID;
bh=zEzD5yerxXfPrn0uP9hcyeowgfg9wdIHGmMA6AM48w4=; b=oJHTwUElEEx9LI3fg5UB4iBBUp
kvYn3OeSArNil6g0H/3AAr8M7d2bqpELlTWjZ9o+b5oyDUWINaqgnDjsF/ck9MLOIX6cSCwHpk2ni
Tt8h8Veo/yPCCvbvz/YJRjyiqL6DOXOYmosFLVuF+n4I168D681f6O3qVLvsapyKvfkfxKjbZk7DA
VxDIEi8x7Nja/ZXYme+yVeqqZA0zB3zAGJwLGxo5vOtt24joCOz0oVJcLpqtYmE+GA+e+8Zdpcx7D
0ClMlykxltoy0kHNfGtF9gcECH5/OVbTfK5IYVSLK1RCtkbped9L9Ds17VGUc0EmQPaaPRahqU0Oe
DOcCDQNA==;
Authentication-Results: synaq.com; iprev=fail smtp.remote-ip=197.90.32.26;
auth=pass (PLAIN) smtp.auth=frank@chagford.com;
arc=none
User-Agent: Mozilla Thunderbird
Content-Language: en-US
In-Reply-To: <8a7231df-c30c-4fa6-9daa-546b3453b547@thomasnyberg.com>
X-Red-Router: yes
X-SYNAQ-Pinpoint-Information: Please contact SYNAQ for more information
X-SYNAQ-Pinpoint-ID: 1rnbpy-0004A2-Fh
X-SYNAQ-Pinpoint: No virus infections found
X-Pinpoint-From: frank@chagford.com
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: General discussion list for the Python programming language
<python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
<mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
<mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <7af73918-8209-4393-8aa2-140940567456@chagford.com>
X-Mailman-Original-References: <8a7231df-c30c-4fa6-9daa-546b3453b547@thomasnyberg.com>

by: Frank Millman - Fri, 22 Mar 2024 10:09 UTC

On 2024-03-20 10:22 AM, Thomas Nyberg via Python-list wrote:
>
> Hello,
>
> I have a simple (and not working) example of what I'm trying to do. This
> is a simplified version of what I'm trying to achieve (obviously the
> background workers and finalizer functions will do more later):
>
> `app.py`
>
> ```
> import asyncio
> import threading
> import time
> from queue import Queue
>
> from flask import Flask
>
> in_queue = Queue()
> out_queue = Queue()
>
>
> def worker():
>     print("worker started running")
>     while True:
>         future = in_queue.get()
>         print(f"worker got future: {future}")
>         time.sleep(5)
>         print("worker sleeped")
>         out_queue.put(future)
>
>
> def finalizer():
>     print("finalizer started running")
>     while True:
>         future = out_queue.get()
>         print(f"finalizer got future: {future}")
>         future.set_result("completed")
>         print("finalizer set result")
>
>
> threading.Thread(target=worker, daemon=True).start()
> threading.Thread(target=finalizer, daemon=True).start()
>
> app = Flask(__name__)
>
>
> @app.route("/")
> async def root():
>     future = asyncio.get_event_loop().create_future()
>     in_queue.put(future)
>     print(f"root put future: {future}")
>     result = await future
>     return result
>
>
> if __name__ == "__main__":
>     app.run()
> ```
>
> If I start up that server, and execute `curl http://localhost:5000`, it
> prints out the following in the server before hanging:
>
> ```
> $ python3 app.py
> worker started running
> finalizer started running
> * Serving Flask app 'app'
> * Debug mode: off
> WARNING: This is a development server. Do not use it in a production
> deployment. Use a production WSGI server instead.
> * Running on http://127.0.0.1:5000
> Press CTRL+C to quit
> root put future: <Future pending>
> worker got future: <Future pending cb=[Task.task_wakeup()]>
> worker sleeped
> finalizer got future: <Future pending cb=[Task.task_wakeup()]>
> finalizer set result
> ```
>
> Judging by what's printing out, the `final result = await future`
> doesn't seem to be happy here.
>
> Maybe someone sees something obvious I'm doing wrong here? I presume I'm
> mixing threads and asyncio in a way I shouldn't be.
>
> Here's some system information (just freshly installed with pip3 install
> flask[async] in a virtual environment for python version 3.11.2):
>
> ```
> $ uname -a
> Linux x1carbon 6.1.0-18-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.76-1
> (2024-02-01) x86_64 GNU/Linux
>
> $ python3 -V
> Python 3.11.2
>
> $ pip3 freeze
> asgiref==3.7.2
> blinker==1.7.0
> click==8.1.7
> Flask==3.0.2
> itsdangerous==2.1.2
> Jinja2==3.1.3
> MarkupSafe==2.1.5
> Werkzeug==3.0.1
> ```
>
> Thanks for any help!
>
> Cheers,
> Thomas

Hi Thomas

I am no expert. However, I do have something similar in my app, and it
works.

I do not use 'await future', I use 'asyncio.wait_for(future)'.

HTH

Frank Millman

"Tell the truth and run." -- Yugoslav proverb

devel / comp.lang.python / Re: Using a background thread with asyncio/futures with flask

Subject	Author
Re: Using a background thread with asyncio/futures with flask	Frank Millman