I want to do the query from with in script based on the interface here [1]. For this purpose, the underlying posting URL must be obtained, say, the URL corresponding to "ITA Settings" button, so that I can make the corresponding query URL and issue the query from the script.

However, I did not find the conversion rules from these buttons to the corresponding URL. Any hints for achieving this aim?

[1] https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen?list=new&what=gen&gnum=10

Regards,
Zhao

On 9/12/2022 5:00 AM, hongy...@gmail.com wrote:
> I want to do the query from with in script based on the interface here [1]. For this purpose, the underlying posting URL must be obtained, say, the URL corresponding to "ITA Settings" button, so that I can make the corresponding query URL and issue the query from the script.
>
> However, I did not find the conversion rules from these buttons to the corresponding URL. Any hints for achieving this aim?
>
> [1] https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen?list=new&what=gen&gnum=10
>
> Regards,
> Zhao

You didn't say what you want to query. Are you trying to download
entire sections of the Bilbao Crystallographic Server? Maybe the admins
will give you access to the data.

* this link: https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen
brings up the table of space group symbols.

* choose say #7: Pc

* now click ITA Settings, then choose the last entry "P c 1 1" and it
loads:

https://www.cryst.ehu.es/cgi-bin/cryst/programs//nph-trgen?gnum=007&what=gp&trmat=b,-a-c,c&unconv=P%20c%201%201&from=ita

You might be able to fool around with that URL and substitute values and
get back the data you want (in HTML) via Python. Do you really want
HTML results?

Hit Ctrl+U to see the source HTML of a webpage

Right-click or hit Ctrl + Shift + C to inspect the individual elements
of the page

On Tuesday, September 13, 2022 at 4:20:12 AM UTC+8, DFS wrote:
> On 9/12/2022 5:00 AM, hongy...@gmail.com wrote:
> > I want to do the query from with in script based on the interface here [1]. For this purpose, the underlying posting URL must be obtained, say, the URL corresponding to "ITA Settings" button, so that I can make the corresponding query URL and issue the query from the script.
> >
> > However, I did not find the conversion rules from these buttons to the corresponding URL. Any hints for achieving this aim?
> >
> > [1] https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen?list=new&what=gen&gnum=10
> >
> > Regards,
> > Zhao
> You didn't say what you want to query. Are you trying to download
> entire sections of the Bilbao Crystallographic Server?

I am engaged in some related research and need some specific data used by BCS server.

> Maybe the admins will give you access to the data.

I don't think they will provide such convenience to researchers who have no cooperative relationship with them.

> * this link: https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen
> brings up the table of space group symbols.
>
> * choose say #7: Pc
>
> * now click ITA Settings, then choose the last entry "P c 1 1" and it
> loads:
>
> https://www.cryst.ehu.es/cgi-bin/cryst/programs//nph-trgen?gnum=007&what=gp&trmat=b,-a-c,c&unconv=P%20c%201%201&from=ita

Not only that, but I want to obtain all such URLs programmatically!
> You might be able to fool around with that URL and substitute values and
> get back the data you want (in HTML) via Python. Do you really want
> HTML results?
>
> Hit Ctrl+U to see the source HTML of a webpage
>
> Right-click or hit Ctrl + Shift + C to inspect the individual elements
> of the page

For batch operations, all these manual methods are inefficient.

Best Regards,
Zhao

On 9/13/2022 3:46 AM, hongy...@gmail.com wrote:
> On Tuesday, September 13, 2022 at 4:20:12 AM UTC+8, DFS wrote:
>> On 9/12/2022 5:00 AM, hongy...@gmail.com wrote:
>>> I want to do the query from with in script based on the interface here [1]. For this purpose, the underlying posting URL must be obtained, say, the URL corresponding to "ITA Settings" button, so that I can make the corresponding query URL and issue the query from the script.
>>>
>>> However, I did not find the conversion rules from these buttons to the corresponding URL. Any hints for achieving this aim?
>>>
>>> [1] https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen?list=new&what=gen&gnum=10
>>>
>>> Regards,
>>> Zhao
>> You didn't say what you want to query. Are you trying to download
>> entire sections of the Bilbao Crystallographic Server?
>
> I am engaged in some related research and need some specific data used by BCS server.

What specific data?

Is it available elsewhere?

>> Maybe the admins will give you access to the data.
>
> I don't think they will provide such convenience to researchers who have no cooperative relationship with them.

You can try. Tell the admins what data you want, and ask them for the
easiest way to get it.

>> * this link: https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen
>> brings up the table of space group symbols.
>>
>> * choose say #7: Pc
>>
>> * now click ITA Settings, then choose the last entry "P c 1 1" and it
>> loads:
>>
>> https://www.cryst.ehu.es/cgi-bin/cryst/programs//nph-trgen?gnum=007&what=gp&trmat=b,-a-c,c&unconv=P%20c%201%201&from=ita
>
> Not only that, but I want to obtain all such URLs programmatically!
>
>> You might be able to fool around with that URL and substitute values and
>> get back the data you want (in HTML) via Python. Do you really want
>> HTML results?
>>
>> Hit Ctrl+U to see the source HTML of a webpage
>>
>> Right-click or hit Ctrl + Shift + C to inspect the individual elements
>> of the page
>
> For batch operations, all these manual methods are inefficient.

Yes, but I don't think you'll be able to retrieve the URLs
programmatically. The JavaScript code doesn't put them in the HTML
result, except for that one I showed you, which seems like a mistake on
their part.

So you'll have to figure out the search fields, and your python program
will have to cycle through the search values:

Sample from above
gnum = 007
what = gp
trmat = b,-a-c,c
unconv = P c 1 1
from = ita

wBase = "https://www.cryst.ehu.es/cgi-bin/cryst/programs//nph-trgen"
wGnum = "?gnum=" + findgnum
wWhat = "&what=" + findWhat
wTrmat = "&trmat=" + findTrmat
wUnconv = "&unconv=" + findUnconv
wFrom = "&from=" + findFrom
webpage = wBase + wGnum + wWhat + wTrmat + wUnconv + wFrom

Then if that returns a hit, you'll have to parse the resulting HTML and
extract the exact data you want.

I did something similar a while back using the requests and lxml libraries
----------------------------------------------------------------
#build url
wBase = "http://www.usdirectory.com"
wForm = "/ypr.aspx?fromform=qsearch"
wKeyw = "&qhqn=" + keyw
wCityZip = "&qc=" + cityzip
wState = "&qs=" + state
wDist = "&rg=" + str(miles)
wSort = "&sb=a2z" #sort alpha
wPage = "&ap=" #used with the results page number
webpage = wBase + wForm + wKeyw + wCityZip + wState + wDist

#open URL
page = requests.get(webpage)
tree = html.fromstring(page.content)

#no matches
matches = tree.xpath('//strong/text()')
if passNbr == 1 and ("No results were found" in str(matches)):
print "No results found for that search"
exit(0)
----------------------------------------------------------------

2.x code file: https://file.io/VdptORSKh5CN

> Best Regards,
> Zhao

On Tuesday, September 13, 2022 at 9:33:20 PM UTC+8, DFS wrote:
> On 9/13/2022 3:46 AM, hongy...@gmail.com wrote:
> > On Tuesday, September 13, 2022 at 4:20:12 AM UTC+8, DFS wrote:
> >> On 9/12/2022 5:00 AM, hongy...@gmail.com wrote:
> >>> I want to do the query from with in script based on the interface here [1]. For this purpose, the underlying posting URL must be obtained, say, the URL corresponding to "ITA Settings" button, so that I can make the corresponding query URL and issue the query from the script.
> >>>
> >>> However, I did not find the conversion rules from these buttons to the corresponding URL. Any hints for achieving this aim?
> >>>
> >>> [1] https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen?list=new&what=gen&gnum=10
> >>>
> >>> Regards,
> >>> Zhao
> >> You didn't say what you want to query. Are you trying to download
> >> entire sections of the Bilbao Crystallographic Server?
> >
> > I am engaged in some related research and need some specific data used by BCS server.
> What specific data?

All the data corresponding to the total catalog here:
https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen
> Is it available elsewhere?

This is an internationally recognized authoritative data source in this field. Data from other places, even if there are readily available electronic versions, are basically taken from here and are not comprehensive.

> >> Maybe the admins will give you access to the data.
> >
> > I don't think they will provide such convenience to researchers who have no cooperative relationship with them.
> You can try. Tell the admins what data you want, and ask them for the
> easiest way to get it.
> >> * this link: https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen
> >> brings up the table of space group symbols.
> >>
> >> * choose say #7: Pc
> >>
> >> * now click ITA Settings, then choose the last entry "P c 1 1" and it
> >> loads:
> >>
> >> https://www.cryst.ehu.es/cgi-bin/cryst/programs//nph-trgen?gnum=007&what=gp&trmat=b,-a-c,c&unconv=P%20c%201%201&from=ita
> >
> > Not only that, but I want to obtain all such URLs programmatically!
> >
> >> You might be able to fool around with that URL and substitute values and
> >> get back the data you want (in HTML) via Python. Do you really want
> >> HTML results?
> >>
> >> Hit Ctrl+U to see the source HTML of a webpage
> >>
> >> Right-click or hit Ctrl + Shift + C to inspect the individual elements
> >> of the page
> >
> > For batch operations, all these manual methods are inefficient.
> Yes, but I don't think you'll be able to retrieve the URLs
> programmatically. The JavaScript code doesn't put them in the HTML
> result, except for that one I showed you, which seems like a mistake on
> their part.
>
> So you'll have to figure out the search fields, and your python program
> will have to cycle through the search values:
>
> Sample from above
> gnum = 007
> what = gp
> trmat = b,-a-c,c
> unconv = P c 1 1
> from = ita

The problem is that I must first get all possible combinations of these variables.
> wBase = "https://www.cryst.ehu.es/cgi-bin/cryst/programs//nph-trgen"
> wGnum = "?gnum=" + findgnum
> wWhat = "&what=" + findWhat
> wTrmat = "&trmat=" + findTrmat
> wUnconv = "&unconv=" + findUnconv
> wFrom = "&from=" + findFrom
> webpage = wBase + wGnum + wWhat + wTrmat + wUnconv + wFrom
>
> Then if that returns a hit, you'll have to parse the resulting HTML and
> extract the exact data you want.
>
>
>
> I did something similar a while back using the requests and lxml libraries
> ----------------------------------------------------------------
> #build url
> wBase = "http://www.usdirectory.com"
> wForm = "/ypr.aspx?fromform=qsearch"
> wKeyw = "&qhqn=" + keyw
> wCityZip = "&qc=" + cityzip
> wState = "&qs=" + state
> wDist = "&rg=" + str(miles)
> wSort = "&sb=a2z" #sort alpha
> wPage = "&ap=" #used with the results page number
> webpage = wBase + wForm + wKeyw + wCityZip + wState + wDist
>
> #open URL
> page = requests.get(webpage)
> tree = html.fromstring(page.content)
>
> #no matches
> matches = tree.xpath('//strong/text()')
> if passNbr == 1 and ("No results were found" in str(matches)):
> print "No results found for that search"
> exit(0)
> ----------------------------------------------------------------
>
>
>
> 2.x code file: https://file.io/VdptORSKh5CN
>
>
>
> > Best Regards,
> > Zhao

On 9/13/2022 7:29 PM, hongy...@gmail.com wrote:
> On Tuesday, September 13, 2022 at 9:33:20 PM UTC+8, DFS wrote:
>> On 9/13/2022 3:46 AM, hongy...@gmail.com wrote:
>>> On Tuesday, September 13, 2022 at 4:20:12 AM UTC+8, DFS wrote:
>>>> On 9/12/2022 5:00 AM, hongy...@gmail.com wrote:
>>>>> I want to do the query from with in script based on the interface here [1]. For this purpose, the underlying posting URL must be obtained, say, the URL corresponding to "ITA Settings" button, so that I can make the corresponding query URL and issue the query from the script.
>>>>>
>>>>> However, I did not find the conversion rules from these buttons to the corresponding URL. Any hints for achieving this aim?
>>>>>
>>>>> [1] https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen?list=new&what=gen&gnum=10" rel="nofollow" target="_blank">https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen?list=new&what=gen&gnum=10
>>>>>
>>>>> Regards,
>>>>> Zhao
>>>> You didn't say what you want to query. Are you trying to download
>>>> entire sections of the Bilbao Crystallographic Server?
>>>
>>> I am engaged in some related research and need some specific data used by BCS server.
>> What specific data?
>
> All the data corresponding to the total catalog here:
> https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen
>
>> Is it available elsewhere?
>
> This is an internationally recognized authoritative data source in this field. Data from other places, even if there are readily available electronic versions, are basically taken from here and are not comprehensive.
>
>>>> Maybe the admins will give you access to the data.
>>>
>>> I don't think they will provide such convenience to researchers who have no cooperative relationship with them.
>> You can try. Tell the admins what data you want, and ask them for the
>> easiest way to get it.
>>>> * this link: https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen
>>>> brings up the table of space group symbols.
>>>>
>>>> * choose say #7: Pc
>>>>
>>>> * now click ITA Settings, then choose the last entry "P c 1 1" and it
>>>> loads:
>>>>
>>>> https://www.cryst.ehu.es/cgi-bin/cryst/programs//nph-trgen?gnum=007&what=gp&trmat=b,-a-c,c&unconv=P%20c%201%201&from=ita
>>>
>>> Not only that, but I want to obtain all such URLs programmatically!
>>>
>>>> You might be able to fool around with that URL and substitute values and
>>>> get back the data you want (in HTML) via Python. Do you really want
>>>> HTML results?
>>>>
>>>> Hit Ctrl+U to see the source HTML of a webpage
>>>>
>>>> Right-click or hit Ctrl + Shift + C to inspect the individual elements
>>>> of the page
>>>
>>> For batch operations, all these manual methods are inefficient.
>> Yes, but I don't think you'll be able to retrieve the URLs
>> programmatically. The JavaScript code doesn't put them in the HTML
>> result, except for that one I showed you, which seems like a mistake on
>> their part.
>>
>> So you'll have to figure out the search fields, and your python program
>> will have to cycle through the search values:
>>
>> Sample from above
>> gnum = 007
>> what = gp
>> trmat = b,-a-c,c
>> unconv = P c 1 1
>> from = ita
>
> The problem is that I must first get all possible combinations of these variables.

Shouldn't be too hard, but I've never done some of these things and have
no code for you:

space group number = gnum = 1 to 230

* use python to put each of those values, one at a time, into the group
number field on the webpage

* use python to simulate a button click of the ITA Settings button

* it should load the HTML of the list of ITA settings for that space group

* use python to parse the HTML and extract each of the ITA settings.
The line of HTML has 'ITA number' in it. Find each of the 'href' values
in the line(s).

Real HTML from ITA Settings for space group 10:

<tr><th bgcolor="#bbbbbb">ITA number</th> <th
bgcolor="#bbbbbb">Setting</th></tr><tr><td align="center"
bgcolor="#f0f0f0">10</td> <td align="center"><a
href="/cgi-bin/cryst/programs//nph-getgen?gnum=010&what=gp"><i>P</i> 1
2/<i>m</i> 1</a></td></tr><tr><td align="center"
bgcolor="#f0f0f0">10</td> <td align="center"><a
href="/cgi-bin/cryst/programs//nph-trgen?gnum=010&what=gp&trmat=c,a,b&unconv=P
1 1 2/m&from=ita"><i>P</i> 1 1 2/<i>m</i></a></td></tr><tr><td
align="center" bgcolor="#f0f0f0">10</td> <td align="center"><a
href="/cgi-bin/cryst/programs//nph-trgen?gnum=010&what=gp&trmat=b,c,a&unconv=P
2/m 1 1&from=ita"><i>P</i> 2/<i>m</i> 1 1</a></td></tr></table>
</center>

If you parse it right you'll have these addresses:

"/cgi-bin/cryst/programs//nph-getgen?gnum=010&what=gp"

"/cgi-bin/cryst/programs//nph-trgen?gnum=010&what=gp&trmat=c,a,b&unconv=P 1
1 2/m&from=ita"

"/cgi-bin/cryst/programs//nph-trgen?gnum=010&what=gp&trmat=b,c,a&unconv=P 2/m
1 1&from=ita"

Then you can parse each of these addresses and build a master list of
the valid combinations of:

gnum, what, trmat, unconv, from

Check into the lxml library, and the 'etree' class. https://lxml.de

You can also search gen.lib.rus.ec for the crystallography volumes, and
maybe cut and paste data from them.

>> wBase = "https://www.cryst.ehu.es/cgi-bin/cryst/programs//nph-trgen"
>> wGnum = "?gnum=" + findgnum
>> wWhat = "&what=" + findWhat
>> wTrmat = "&trmat=" + findTrmat
>> wUnconv = "&unconv=" + findUnconv
>> wFrom = "&from=" + findFrom
>> webpage = wBase + wGnum + wWhat + wTrmat + wUnconv + wFrom
>>
>> Then if that returns a hit, you'll have to parse the resulting HTML and
>> extract the exact data you want.
>>
>>
>>
>> I did something similar a while back using the requests and lxml libraries
>> ----------------------------------------------------------------
>> #build url
>> wBase = "http://www.usdirectory.com"
>> wForm = "/ypr.aspx?fromform=qsearch"
>> wKeyw = "&qhqn=" + keyw
>> wCityZip = "&qc=" + cityzip
>> wState = "&qs=" + state
>> wDist = "&rg=" + str(miles)
>> wSort = "&sb=a2z" #sort alpha
>> wPage = "&ap=" #used with the results page number
>> webpage = wBase + wForm + wKeyw + wCityZip + wState + wDist
>>
>> #open URL
>> page = requests.get(webpage)
>> tree = html.fromstring(page.content)
>>
>> #no matches
>> matches = tree.xpath('//strong/text()')
>> if passNbr == 1 and ("No results were found" in str(matches)):
>> print "No results found for that search"
>> exit(0)
>> ----------------------------------------------------------------
>>
>>
>>
>> 2.x code file: https://file.io/VdptORSKh5CN
>>
>>
>>
>>> Best Regards,
>>> Zhao

You're welcome.

On Wednesday, September 14, 2022 at 10:41:32 AM UTC+8, DFS wrote:
> On 9/13/2022 7:29 PM, hongy...@gmail.com wrote:
> > On Tuesday, September 13, 2022 at 9:33:20 PM UTC+8, DFS wrote:
> >> On 9/13/2022 3:46 AM, hongy...@gmail.com wrote:
> >>> On Tuesday, September 13, 2022 at 4:20:12 AM UTC+8, DFS wrote:
> >>>> On 9/12/2022 5:00 AM, hongy...@gmail.com wrote:
> >>>>> I want to do the query from with in script based on the interface here [1]. For this purpose, the underlying posting URL must be obtained, say, the URL corresponding to "ITA Settings" button, so that I can make the corresponding query URL and issue the query from the script.
> >>>>>
> >>>>> However, I did not find the conversion rules from these buttons to the corresponding URL. Any hints for achieving this aim?
> >>>>>
> >>>>> [1] https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen?list=new&what=gen&gnum=10" rel="nofollow" target="_blank">https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen?list=new&what=gen&gnum=10
> >>>>>
> >>>>> Regards,
> >>>>> Zhao
> >>>> You didn't say what you want to query. Are you trying to download
> >>>> entire sections of the Bilbao Crystallographic Server?
> >>>
> >>> I am engaged in some related research and need some specific data used by BCS server.
> >> What specific data?
> >
> > All the data corresponding to the total catalog here:
> > https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen
> >
> >> Is it available elsewhere?
> >
> > This is an internationally recognized authoritative data source in this field. Data from other places, even if there are readily available electronic versions, are basically taken from here and are not comprehensive.
> >
> >>>> Maybe the admins will give you access to the data.
> >>>
> >>> I don't think they will provide such convenience to researchers who have no cooperative relationship with them.
> >> You can try. Tell the admins what data you want, and ask them for the
> >> easiest way to get it.
> >>>> * this link: https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen
> >>>> brings up the table of space group symbols.
> >>>>
> >>>> * choose say #7: Pc
> >>>>
> >>>> * now click ITA Settings, then choose the last entry "P c 1 1" and it
> >>>> loads:
> >>>>
> >>>> https://www.cryst.ehu.es/cgi-bin/cryst/programs//nph-trgen?gnum=007&what=gp&trmat=b,-a-c,c&unconv=P%20c%201%201&from=ita
> >>>
> >>> Not only that, but I want to obtain all such URLs programmatically!
> >>>
> >>>> You might be able to fool around with that URL and substitute values and
> >>>> get back the data you want (in HTML) via Python. Do you really want
> >>>> HTML results?
> >>>>
> >>>> Hit Ctrl+U to see the source HTML of a webpage
> >>>>
> >>>> Right-click or hit Ctrl + Shift + C to inspect the individual elements
> >>>> of the page
> >>>
> >>> For batch operations, all these manual methods are inefficient.
> >> Yes, but I don't think you'll be able to retrieve the URLs
> >> programmatically. The JavaScript code doesn't put them in the HTML
> >> result, except for that one I showed you, which seems like a mistake on
> >> their part.
> >>
> >> So you'll have to figure out the search fields, and your python program
> >> will have to cycle through the search values:
> >>
> >> Sample from above
> >> gnum = 007
> >> what = gp
> >> trmat = b,-a-c,c
> >> unconv = P c 1 1
> >> from = ita
> >
> > The problem is that I must first get all possible combinations of these variables.
> Shouldn't be too hard, but I've never done some of these things and have
> no code for you:
>
> space group number = gnum = 1 to 230
>
> * use python to put each of those values, one at a time, into the group
> number field on the webpage
>
> * use python to simulate a button click of the ITA Settings button
>
> * it should load the HTML of the list of ITA settings for that space group

This is the trickiest part of the problem. For this purpose, Vladimir gave the following suggestion here [1]:

This is trivial with Selenium. Highlight the element by XPATH and extract URL with .get_attribute("href")

[1] https://discuss.python.org/t/obtain-the-query-interface-url-of-bcs-server/18996/2?u=hongyi-zhao

> * use python to parse the HTML and extract each of the ITA settings.
> The line of HTML has 'ITA number' in it. Find each of the 'href' values
> in the line(s).
>
> Real HTML from ITA Settings for space group 10:
>
> <tr><th bgcolor="#bbbbbb">ITA number</th> <th
> bgcolor="#bbbbbb">Setting</th></tr><tr><td align="center"
> bgcolor="#f0f0f0">10</td> <td align="center"><a
> href="/cgi-bin/cryst/programs//nph-getgen?gnum=010&what=gp"><i>P</i> 1
> 2/<i>m</i> 1</a></td></tr><tr><td align="center"
> bgcolor="#f0f0f0">10</td> <td align="center"><a
> href="/cgi-bin/cryst/programs//nph-trgen?gnum=010&what=gp&trmat=c,a,b&unconv=P
> 1 1 2/m&from=ita"><i>P</i> 1 1 2/<i>m</i></a></td></tr><tr><td
> align="center" bgcolor="#f0f0f0">10</td> <td align="center"><a
> href="/cgi-bin/cryst/programs//nph-trgen?gnum=010&what=gp&trmat=b,c,a&unconv=P
> 2/m 1 1&from=ita"><i>P</i> 2/<i>m</i> 1 1</a></td></tr></table>
> </center>
>
> If you parse it right you'll have these addresses:
>
> "/cgi-bin/cryst/programs//nph-getgen?gnum=010&what=gp"
>
> "/cgi-bin/cryst/programs//nph-trgen?gnum=010&what=gp&trmat=c,a,b&unconv=P 1
> 1 2/m&from=ita"
>
> "/cgi-bin/cryst/programs//nph-trgen?gnum=010&what=gp&trmat=b,c,a&unconv=P 2/m
> 1 1&from=ita"
>
>
> Then you can parse each of these addresses and build a master list of
> the valid combinations of:
>
> gnum, what, trmat, unconv, from
>
> Check into the lxml library, and the 'etree' class. https://lxml.de
>
> You can also search gen.lib.rus.ec for the crystallography volumes, and
> maybe cut and paste data from them.

The relevant search results are as follows:

http://libgen.rs/search.php?&req=International+Tables+for+Crystallography&phrase=1&view=simple&column=def&sort=year&sortmode=DESC

Best,
Zhao