Hi,

I am using SQLAlchemy to extract some rows from a table of 'events'.
From the call to the DB I get a list of objects of the type

sqlalchemy.orm.state.InstanceState

I would like to print these rows to the terminal using the 'tabulate'
package, the documentation for which says

The module provides just one function, tabulate, which takes a list of
lists or another tabular data type as the first argument, and outputs
a nicely formatted plain-text table

So as I understand it, I need to convert the InstanceState-objects to,
say, dicts, in order to print them. However I also want to remove one
of the keys from the output and assumed I could just pop it off each
event dict, thus:

event_dicts = [vars(e) for e in events]
print(type(event_dicts[0]))
event_dicts = [e.pop('_sa_instance_state', None) for e in event_dicts]
print(type(event_dicts[0]))

However, this prints

<class 'dict'>
<class 'sqlalchemy.orm.state.InstanceState'>

If I comment out the third line, which pops the unwanted key, I get

<class 'dict'>
<class 'dict'>

Why does popping one of the keys cause the elements of the list to
revert back to their original class?

Cheers,

Loris

--
This signature is currently under constuction.

Loris Bennett wrote at 2024-3-21 10:56 +0100:
> ...
>So as I understand it, I need to convert the InstanceState-objects to,
>say, dicts, in order to print them. However I also want to remove one
>of the keys from the output and assumed I could just pop it off each
>event dict, thus:
>
> event_dicts = [vars(e) for e in events]
> print(type(event_dicts[0]))
> event_dicts = [e.pop('_sa_instance_state', None) for e in event_dicts]
> print(type(event_dicts[0]))
>
>However, this prints
>
> <class 'dict'>

`vars` typically returns a `dict`.

> <class 'sqlalchemy.orm.state.InstanceState'>
This is what you have popped.
>
>If I comment out the third line, which pops the unwanted key, I get
Then you do not change `event_dicts`.

You problem likely is:
`pop` does not return the `dict` after the removal of a key
but the removed value.

Loris Bennett wrote:
> Hi,
>
> I am using SQLAlchemy to extract some rows from a table of 'events'.
> From the call to the DB I get a list of objects of the type
>
> sqlalchemy.orm.state.InstanceState
>
> I would like to print these rows to the terminal using the 'tabulate'
> package, the documentation for which says
>
> The module provides just one function, tabulate, which takes a list of
> lists or another tabular data type as the first argument, and outputs
> a nicely formatted plain-text table
>
> So as I understand it, I need to convert the InstanceState-objects to,
> say, dicts, in order to print them. However I also want to remove one
> of the keys from the output and assumed I could just pop it off each
> event dict, thus:
>
> event_dicts = [vars(e) for e in events]
> print(type(event_dicts[0]))
> event_dicts = [e.pop('_sa_instance_state', None) for e in event_dicts]
> print(type(event_dicts[0]))

vars() returns the __dict__ attribute of the object. It may not be a
good idea to modify that dictionary directly (it will also affect the
object), although it might be OK if you're not going to do anything else
with the original objects. To be safer, you could copy the event objects:
event_dicts = [dict(vars(e)) for e in events]
or:
event_dicts = [vars(e).copy()]

> However, this prints
>
> <class 'dict'>
> <class 'sqlalchemy.orm.state.InstanceState'>
>
> If I comment out the third line, which pops the unwanted key, I get
>
> <class 'dict'>
> <class 'dict'>
>
> Why does popping one of the keys cause the elements of the list to
> revert back to their original class?

As Dieter pointed out, the main problem here is that pop() returns the
value removed, not the dictionary with the rest of the values. You
probably want something more like:
for e in event_dicts:
del e['_sa_instance_state']
(There's not really any point popping the value if you're not going to
do anything with it - just delete the key from the dictionary)

--
Mark.

Mark Bourne <nntp.mbourne@spamgourmet.com> writes:

> Loris Bennett wrote:
>> Hi,
>> I am using SQLAlchemy to extract some rows from a table of 'events'.
>> From the call to the DB I get a list of objects of the type
>> sqlalchemy.orm.state.InstanceState
>> I would like to print these rows to the terminal using the
>> 'tabulate'
>> package, the documentation for which says
>> The module provides just one function, tabulate, which takes a
>> list of
>> lists or another tabular data type as the first argument, and outputs
>> a nicely formatted plain-text table
>> So as I understand it, I need to convert the InstanceState-objects
>> to,
>> say, dicts, in order to print them. However I also want to remove one
>> of the keys from the output and assumed I could just pop it off each
>> event dict, thus:
>> event_dicts = [vars(e) for e in events]
>> print(type(event_dicts[0]))
>> event_dicts = [e.pop('_sa_instance_state', None) for e in event_dicts]
>> print(type(event_dicts[0]))
>
> vars() returns the __dict__ attribute of the object. It may not be a
> good idea to modify that dictionary directly (it will also affect the
> object), although it might be OK if you're not going to do anything
> else with the original objects. To be safer, you could copy the event
> objects:
> event_dicts = [dict(vars(e)) for e in events]
> or:
> event_dicts = [vars(e).copy()]

Thanks for making this clear to me. However, in the end I actually
decided to use the list comprehension without either 'dict()' or
'vars(). Instead I just select the keys I want and so don't need to pop
the unwanted key later and can simultaneously tweak the names of the
key for better printing to the terminal.

>> However, this prints
>> <class 'dict'>
>> <class 'sqlalchemy.orm.state.InstanceState'>
>> If I comment out the third line, which pops the unwanted key, I get
>> <class 'dict'>
>> <class 'dict'>
>> Why does popping one of the keys cause the elements of the list to
>> revert back to their original class?
>
> As Dieter pointed out, the main problem here is that pop() returns the
> value removed, not the dictionary with the rest of the values. You
> probably want something more like:
> for e in event_dicts:
> del e['_sa_instance_state']
> (There's not really any point popping the value if you're not going to
> do anything with it - just delete the key from the dictionary)

Yes, I was mistakenly thinking that the popping the element would leave
me with the dict minus the popped key-value pair. Seem like there is no
such function.

Cheers,

Loris
--
This signature is currently under constuction.

On 2024-03-22, Loris Bennett via Python-list <python-list@python.org> wrote:

> Yes, I was mistakenly thinking that the popping the element would
> leave me with the dict minus the popped key-value pair.

It does.

> Seem like there is no such function.

Yes, there is. You can do that with either pop or del:

>>> d = {'a':1, 'b':2, 'c':3}
>>> d
{'a': 1, 'b': 2, 'c': 3}
>>> d.pop('b')
2
>>> d
{'a': 1, 'c': 3}

>>> d = {'a':1, 'b':2, 'c':3}
>>> del d['b']
>>> d
{'a': 1, 'c': 3}

In both cases, you're left with the dict minus the key/value pair.

In the first case, the deleted value printed by the REPL because it
was returned by the expression "d.pop('b')" (a method call).

In the second case is no value shown by the REPL because "del d['b']"
is a statement not an expression.

Loris wrote:

"Yes, I was mistakenly thinking that the popping the element would leave
me with the dict minus the popped key-value pair. Seem like there is no
such function."

Others have tried to explain and pointed out you can del and then use the
changed dict.

But consider the odd concept of writing your own trivial function.

def remaining(adict, anitem):
_ = adict.pop(anitem)
# alternatively duse del on dict and item
return adict

>>> remaining({"first": 1, "second": 2, "third": 3}, "second")
{'first': 1, 'third': 3}

Or do you want to be able to call it as in dict.remaining(key) by
subclassing your own variant of dict and adding a similar method?

<avi.e.gross@gmail.com> writes:

> Loris wrote:
>
> "Yes, I was mistakenly thinking that the popping the element would leave
> me with the dict minus the popped key-value pair. Seem like there is no
> such function."
>
> Others have tried to explain and pointed out you can del and then use the
> changed dict.
>
> But consider the odd concept of writing your own trivial function.
>
> def remaining(adict, anitem):
> _ = adict.pop(anitem)
> # alternatively duse del on dict and item
> return adict
>
>
>>>> remaining({"first": 1, "second": 2, "third": 3}, "second")
> {'first': 1, 'third': 3}
>
>
> Or do you want to be able to call it as in dict.remaining(key) by
> subclassing your own variant of dict and adding a similar method?

No, 'del' does indeed do what I wanted, although I have now decided I
want something else :-) Nevertheless it is good to know that 'del'
exists, so that I don't have to reinvent it.

Cheers,

Loris

--
This signature is currently under constuction.

Grant Edwards <grant.b.edwards@gmail.com> writes:

> On 2024-03-22, Loris Bennett via Python-list <python-list@python.org> wrote:
>
>> Yes, I was mistakenly thinking that the popping the element would
>> leave me with the dict minus the popped key-value pair.
>
> It does.

Indeed, but I was thinking in the context of

dict_list = [d.pop('a') for d in dict_list]

and incorrectly expecting to get a list of 'd' without key 'a', instead
of a list of the 'd['a]'.

>> Seem like there is no such function.
>
> Yes, there is. You can do that with either pop or del:
>
> >>> d = {'a':1, 'b':2, 'c':3}
> >>> d
> {'a': 1, 'b': 2, 'c': 3}
> >>> d.pop('b')
> 2
> >>> d
> {'a': 1, 'c': 3}
>
>
> >>> d = {'a':1, 'b':2, 'c':3}
> >>> del d['b']
> >>> d
> {'a': 1, 'c': 3}
>
> In both cases, you're left with the dict minus the key/value pair.
>
> In the first case, the deleted value printed by the REPL because it
> was returned by the expression "d.pop('b')" (a method call).
>
> In the second case is no value shown by the REPL because "del d['b']"
> is a statement not an expression.

Thanks for pointing out 'del'. My main problem, however, was failing to
realise that the list comprehension is populated by the return value of
the 'pop', not the popped dict.

Cheers,

Loris

--
This signature is currently under constuction.

On 25/03/2024 01.56, Loris Bennett wrote:
> Grant Edwards <grant.b.edwards@gmail.com> writes:
>
>> On 2024-03-22, Loris Bennett via Python-list <python-list@python.org> wrote:
>>
>>> Yes, I was mistakenly thinking that the popping the element would
>>> leave me with the dict minus the popped key-value pair.
>>
>> It does.
>
> Indeed, but I was thinking in the context of
>
> dict_list = [d.pop('a') for d in dict_list]
>
> and incorrectly expecting to get a list of 'd' without key 'a', instead
> of a list of the 'd['a]'.
I apologize if this has already been mentioned in this thread, but are
you aware of "d.keys()" and "d.values"?

>>> d = {}
>>> d['do'] = 'a deer, a female deer'
>>> d['re'] = 'a drop of golden sunshine'
>>> d['mi'] = 'a name I call myself'
>>> d['fa'] = 'a long, long way to run'
>>> d.keys()
['fa', 'mi', 'do', 're']
>>> d.values()
['a long, long way to run', 'a name I call myself', 'a deer, a female deer', 'a drop of golden sunshine']
>>>

--
Michael F. Stemper
Exodus 22:21

"Michael F. Stemper" <michael.stemper@gmail.com> writes:

> On 25/03/2024 01.56, Loris Bennett wrote:
>> Grant Edwards <grant.b.edwards@gmail.com> writes:
>>
>>> On 2024-03-22, Loris Bennett via Python-list <python-list@python.org> wrote:
>>>
>>>> Yes, I was mistakenly thinking that the popping the element would
>>>> leave me with the dict minus the popped key-value pair.
>>>
>>> It does.
>> Indeed, but I was thinking in the context of
>> dict_list = [d.pop('a') for d in dict_list]
>> and incorrectly expecting to get a list of 'd' without key 'a',
>> instead
>> of a list of the 'd['a]'.
> I apologize if this has already been mentioned in this thread, but are
> you aware of "d.keys()" and "d.values"?
>
> >>> d = {}
> >>> d['do'] = 'a deer, a female deer'
> >>> d['re'] = 'a drop of golden sunshine'
> >>> d['mi'] = 'a name I call myself'
> >>> d['fa'] = 'a long, long way to run'
> >>> d.keys()
> ['fa', 'mi', 'do', 're']
> >>> d.values()
> ['a long, long way to run', 'a name I call myself', 'a deer, a female deer', 'a drop of golden sunshine']
> >>>

Yes, I am, thank you. However, I didn't want either the keys or the
values. Instead I wanted to remove a key within a list comprehension.

Cheers,

Loris

PS: "a drop of golden *sun*" - rhymes with "a long, long way to run"

--
This signature is currently under constuction.

On 2024-03-25, Loris Bennett <loris.bennett@fu-berlin.de> wrote:
> "Michael F. Stemper" <michael.stemper@gmail.com> writes:
>
>> On 25/03/2024 01.56, Loris Bennett wrote:
>>> Grant Edwards <grant.b.edwards@gmail.com> writes:
>>>
>>>> On 2024-03-22, Loris Bennett via Python-list <python-list@python.org> wrote:
>>>>
>>>>> Yes, I was mistakenly thinking that the popping the element would
>>>>> leave me with the dict minus the popped key-value pair.
>>>>
>>>> It does.
>>> Indeed, but I was thinking in the context of
>>> dict_list = [d.pop('a') for d in dict_list]
>>> and incorrectly expecting to get a list of 'd' without key 'a',
>>> instead
>>> of a list of the 'd['a]'.
>> I apologize if this has already been mentioned in this thread, but are
>> you aware of "d.keys()" and "d.values"?
>>
>> >>> d = {}
>> >>> d['do'] = 'a deer, a female deer'
>> >>> d['re'] = 'a drop of golden sunshine'
>> >>> d['mi'] = 'a name I call myself'
>> >>> d['fa'] = 'a long, long way to run'
>> >>> d.keys()
>> ['fa', 'mi', 'do', 're']
>> >>> d.values()
>> ['a long, long way to run', 'a name I call myself', 'a deer, a female deer', 'a drop of golden sunshine']
>> >>>
>
> Yes, I am, thank you. However, I didn't want either the keys or the
> values. Instead I wanted to remove a key within a list comprehension.

Do you mean something like:

[my_dict[key] for key in my_dict if key != 'a']

?

On 2024-03-25, Loris Bennett via Python-list <python-list@python.org> wrote:
> Grant Edwards <grant.b.edwards@gmail.com> writes:
>
>> On 2024-03-22, Loris Bennett via Python-list <python-list@python.org> wrote:
>>
>>> Yes, I was mistakenly thinking that the popping the element would
>>> leave me with the dict minus the popped key-value pair.
>>
>> It does.
>
> Indeed, but I was thinking in the context of
>
> dict_list = [d.pop('a') for d in dict_list]
>
> and incorrectly expecting to get a list of 'd' without key 'a', instead
> of a list of the 'd['a]'.

So when you say "leave me with", you mean "return the same dictionary
with"? There's an important difference between what a function
returns and what global/local state it "leaves you with".

> Thanks for pointing out 'del'. My main problem, however, was
> failing to realise that the list comprehension is populated by the
> return value of the 'pop', not the popped dict.

OK, so perhaps you weren't execting the original dict objects to be
mutated, but rather that the pop method would return a new dict object
without the "popped" element. The whole point of the 'pop method is to
return the popped value, otherwise it wouldn't be needed. The 'del'
statement would suffice.

I am glad, Lori, you found a solution another way.

Actually, Lori, I think you were right in looking for a built-in method that complements pop() by returning everything else other than the item mentioned. There are philosophical and practical considerations that were no doubt considered and a reality that the functionality did exist albeit not in a pipelined format.

Consider a language like LISP which sort of processed lists of things in which a major concept was getting the first item or getting all the rest. Lots of LISP programs had complex structures using CAR() and CDR() nested various ways to say extract the third item as CAR(CDR(CDR(X))) to the point where some commonly used combos became functions with names like CADDR().

There are again many languages where functions or methods are available that include an exclusion variant from a collection perhaps by doing something as simple as using a minus sign to indicate what to remove, as is commonly done in R to remove a column in a data.frame while keeping the order of the remaining columns the same.

Lots of languages had similar concepts about ordered data structures but dictionaries may be a bit of something else and initially in Python were not guaranteed to have any kind of order. Python dicts are more like unordered sets.

So although there remains a concept of not first/rest but this/rest, I suspect there was some thought about the process that ended in deciding not to supply some functionality. When you use pop() on something like a large dictionary, the original is left intact and is ignored and a copy of a single element is made and returned. To do the opposite and return the rest has two choices. One is to make a big copy of the rest of the dictionary and the other is to use del internally and return the modified dict. The functions you could write do the latter.

So why not add one or more methods to do that? Who knows? But I think some may have considered it not needed including some who felt no need for a pipeline method when del would do. Another consideration was the common idiom for iterating on a collection. Besides pop() you can get lists of dictionary entries, keys or values that you can work with and you can even iterate with "for key in dict: ..."

Given how many ways common things can be done, and given that adding too many methods has costs including new users not understanding all the nuanced differences, this fairly useful functionality was left out.

Unfortunately, I think they were wrong here as instead we hear often from people like you who assume things would work other ways. I still think it would be simple enough to have had a .removekeys(keys) that would work in a pipeline to modify the dict by removing one or more items and perhaps another .removevalues(values) but at some point you may keep adding methods nobody ever uses. The reality is that many trivial one-liner comprehensions can easily do many such things using iteration.

But many simply do not work well in pipelined fashion and thus may need to be embedded in a method of your own by subclassing dict or rolling your own.

-----Original Message-----
From: Loris Bennett <loris.bennett@fu-berlin.de>
Sent: Monday, March 25, 2024 2:45 AM
To: avi.e.gross@gmail.com
Cc: python-list@python.org
Subject: Re: Popping key causes dict derived from object to revert to object

<avi.e.gross@gmail.com> writes:

> Loris wrote:
>
> "Yes, I was mistakenly thinking that the popping the element would leave
> me with the dict minus the popped key-value pair. Seem like there is no
> such function."
>
> Others have tried to explain and pointed out you can del and then use the
> changed dict.
>
> But consider the odd concept of writing your own trivial function.
>
> def remaining(adict, anitem):
> _ = adict.pop(anitem)
> # alternatively duse del on dict and item
> return adict
>
>
>>>> remaining({"first": 1, "second": 2, "third": 3}, "second")
> {'first': 1, 'third': 3}
>
>
> Or do you want to be able to call it as in dict.remaining(key) by
> subclassing your own variant of dict and adding a similar method?

No, 'del' does indeed do what I wanted, although I have now decided I
want something else :-) Nevertheless it is good to know that 'del'
exists, so that I don't have to reinvent it.

Cheers,

Loris

--
This signature is currently under constuction.

Lori,

The list comprehension you are thinking of does work if you change things a
bit. But it is not a great idea as a main purpose of a dict is that using a
hash means things are found in linear time. A comprehension iterates on all
values. If you wanted to select just some items to keep in a list, your code
could be modified from:

dict_list = [d.pop('a') for d in dict_list]

to have an IF clause that would specify something like comparing it to the
item you do not want to keep.

But your idiom might be better done to make another dictionaly, not list
with something like:

New_dict = {key:value for key in dict if key != "whatever"}

Or variants on that. It builds a new dictionary, at nontrivial expense, as
compared to using del on an existing dictionary.

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com@python.org> On
Behalf Of Loris Bennett via Python-list
Sent: Monday, March 25, 2024 2:56 AM
To: python-list@python.org
Subject: Re: Popping key causes dict derived from object to revert to object

Grant Edwards <grant.b.edwards@gmail.com> writes:

> On 2024-03-22, Loris Bennett via Python-list <python-list@python.org>
wrote:
>
>> Yes, I was mistakenly thinking that the popping the element would
>> leave me with the dict minus the popped key-value pair.
>
> It does.

Indeed, but I was thinking in the context of

dict_list = [d.pop('a') for d in dict_list]

and incorrectly expecting to get a list of 'd' without key 'a', instead
of a list of the 'd['a]'.

>> Seem like there is no such function.
>
> Yes, there is. You can do that with either pop or del:
>
> >>> d = {'a':1, 'b':2, 'c':3}
> >>> d
> {'a': 1, 'b': 2, 'c': 3}
> >>> d.pop('b')
> 2
> >>> d
> {'a': 1, 'c': 3}
>
>
> >>> d = {'a':1, 'b':2, 'c':3}
> >>> del d['b']
> >>> d
> {'a': 1, 'c': 3}
>
> In both cases, you're left with the dict minus the key/value pair.
>
> In the first case, the deleted value printed by the REPL because it
> was returned by the expression "d.pop('b')" (a method call).
>
> In the second case is no value shown by the REPL because "del d['b']"
> is a statement not an expression.

Thanks for pointing out 'del'. My main problem, however, was failing to
realise that the list comprehension is populated by the return value of
the 'pop', not the popped dict.

Cheers,

Loris

--
This signature is currently under constuction.
--
https://mail.python.org/mailman/listinfo/python-list