Hello,

I'm trying to find a solution in the Web to this issue, when a server sends an HTTL response code 303 (redirect) and the new location (URL) is identical to the original location. Until now I couldn't find a solution in the Web. But obviously there must be a solution since my web browser can handle this.
Any ideas on your side?

Many thanks
Alexandru

On 12/06/2023 14:32, Alexandru wrote:
> I'm trying to find a solution in the Web to this issue, when a server sends an HTTL response code 303 (redirect) and the new location (URL) is identical to the original location. Until now I couldn't find a solution in the Web. But obviously there must be a solution since my web browser can handle this.
> Any ideas on your side?
>
Check carefully. Are the URLs really exactly the same? I remember I had
to look twice when I fetched a URL like "http://www.tcl.tk/about" and
got a redirect to "/about/. The trailing slash makes a difference.

The only way a browser can "handle" a redirect to the exact same URL it
requested is by displaying a page that says something like "The page
isn’t redirecting properly". But I guess that's not what you are
referring to.

Schelte.

Schelte schrieb am Montag, 12. Juni 2023 um 16:02:23 UTC+2:
> On 12/06/2023 14:32, Alexandru wrote:
> > I'm trying to find a solution in the Web to this issue, when a server sends an HTTL response code 303 (redirect) and the new location (URL) is identical to the original location. Until now I couldn't find a solution in the Web. But obviously there must be a solution since my web browser can handle this.
> > Any ideas on your side?
> >
> Check carefully. Are the URLs really exactly the same? I remember I had
> to look twice when I fetched a URL like "http://www.tcl.tk/about" and
> got a redirect to "/about/. The trailing slash makes a difference.
>
> The only way a browser can "handle" a redirect to the exact same URL it
> requested is by displaying a page that says something like "The page
> isn’t redirecting properly". But I guess that's not what you are
> referring to.
>
>
> Schelte.

I double checked, the URLs are identical, including the trailing slash.
This is the URL and the siplified code:

set url https://www.duma-bandzink.com/de/
set token [::http::geturl $url -binary 1]
if {[string index $ncode 0]==3} {
array set meta $state(meta)
if {![info exists meta(Location)] && ![info exists meta(location)]} {
http::cleanup $token
return
}
if {[info exists meta(Location)]} {
set url2 $meta(Location)
} else {
set url2 $meta(location)
}
http::cleanup $token
puts "Old: $url"
puts "New: $url2"
}

Just run the code. Thanks.

On 12/06/2023 16:45, Alexandru wrote:
> I double checked, the URLs are identical, including the trailing slash.
> This is the URL and the siplified code:
>
The code doesn't run as presented. But indeed the URLs are the same.
However, the first response also returns some cookies. Repeating the
request with those cookies results in a 200 OK.

My www package (https://chiselapp.com/user/schelte/repository/www/index)
takes care of redirects and cookies. Fetching the URL that way (you'll
need the latest check-in) works correctly:

package require www
catch {www get https://www.duma-bandzink.com/de/} data info
dict get $info status line
=> HTTP/1.1 200 OK

Schelte.

Schelte schrieb am Montag, 12. Juni 2023 um 17:37:56 UTC+2:
> On 12/06/2023 16:45, Alexandru wrote:
> > I double checked, the URLs are identical, including the trailing slash.
> > This is the URL and the siplified code:
> >
> The code doesn't run as presented. But indeed the URLs are the same.
> However, the first response also returns some cookies. Repeating the
> request with those cookies results in a 200 OK.
>
> My www package (https://chiselapp.com/user/schelte/repository/www/index)
> takes care of redirects and cookies. Fetching the URL that way (you'll
> need the latest check-in) works correctly:
>
> package require www
> catch {www get https://www.duma-bandzink.com/de/} data info
> dict get $info status line
> => HTTP/1.1 200 OK
>
>
> Schelte.

Hi Shelte,

Many thanks, the cookies could be the cause indeed.
I'll take a look at your code.

Regards
Alexandru

Alexandru schrieb am Montag, 12. Juni 2023 um 18:30:40 UTC+2:
> Schelte schrieb am Montag, 12. Juni 2023 um 17:37:56 UTC+2:
> > On 12/06/2023 16:45, Alexandru wrote:
> > > I double checked, the URLs are identical, including the trailing slash.
> > > This is the URL and the siplified code:
> > >
> > The code doesn't run as presented. But indeed the URLs are the same.
> > However, the first response also returns some cookies. Repeating the
> > request with those cookies results in a 200 OK.
> >
> > My www package (https://chiselapp.com/user/schelte/repository/www/index)
> > takes care of redirects and cookies. Fetching the URL that way (you'll
> > need the latest check-in) works correctly:
> >
> > package require www
> > catch {www get https://www.duma-bandzink.com/de/} data info
> > dict get $info status line
> > => HTTP/1.1 200 OK
> >
> >
> > Schelte.
> Hi Shelte,
>
> Many thanks, the cookies could be the cause indeed.
> I'll take a look at your code.
>
> Regards
> Alexandru

Hi Schelte,

How where should I save your package?
I unpacked the ZIP archive to C:\Tcl\lib\www but package require www fails...

On 6/12/2023 11:37 AM, Schelte wrote:
> On 12/06/2023 16:45, Alexandru wrote:
>> I double checked, the URLs are identical, including the trailing slash.
>> This is the URL and the siplified code:
>>
> The code doesn't run as presented. But indeed the URLs are the same.
> However, the first response also returns some cookies. Repeating the
> request with those cookies results in a 200 OK.
>

I see that the cookies are there to make sure the GDPR notice gets
displayed and the user is forced to act on it.

On 12/06/2023 18:41, Alexandru wrote:
> How where should I save your package?
> I unpacked the ZIP archive to C:\Tcl\lib\www but package require www fails...

As it is a Tcl module, you have to put it in one of the directories
returned from [tcl::tm::path list], or add the directory where you put
it to that list, using [tcl::tm::path add C:/Tcl/lib/www].

Schelte.

Schelte schrieb am Montag, 12. Juni 2023 um 21:27:20 UTC+2:
> On 12/06/2023 18:41, Alexandru wrote:
> > How where should I save your package?
> > I unpacked the ZIP archive to C:\Tcl\lib\www but package require www fails...
> As it is a Tcl module, you have to put it in one of the directories
> returned from [tcl::tm::path list], or add the directory where you put
> it to that list, using [tcl::tm::path add C:/Tcl/lib/www].
>
>
> Schelte.

Well, I'm a little bit familiar with the Tcl basics. I put your package as any other package into the recognized directory C:/Tcl/lib.
Should your archive have a pkgIndex.tcl ?
This file is not available so no wonder is not working.
I tried to define the pkgIndex.tcl with this content:

package ifneeded www 2.3 [list source [file join $dir www-2.3.tm]]

This give me the error:

attempt to provide package www 2.3 failed: no version of package www provided

Alexandru schrieb am Dienstag, 13. Juni 2023 um 15:32:13 UTC+2:
> Schelte schrieb am Montag, 12. Juni 2023 um 21:27:20 UTC+2:
> > On 12/06/2023 18:41, Alexandru wrote:
> > > How where should I save your package?
> > > I unpacked the ZIP archive to C:\Tcl\lib\www but package require www fails...
> > As it is a Tcl module, you have to put it in one of the directories
> > returned from [tcl::tm::path list], or add the directory where you put
> > it to that list, using [tcl::tm::path add C:/Tcl/lib/www].
> >
> >
> > Schelte.
> Well, I'm a little bit familiar with the Tcl basics. I put your package as any other package into the recognized directory C:/Tcl/lib.
> Should your archive have a pkgIndex.tcl ?
> This file is not available so no wonder is not working.
> I tried to define the pkgIndex.tcl with this content:
>
> package ifneeded www 2.3 [list source [file join $dir www-2.3.tm]]
>
> This give me the error:
>
> attempt to provide package www 2.3 failed: no version of package www provided

Ah sorry, I think I know what the problem is. The tm are expected in another location than the normal libraries.
Forget my last post.

Alexandru schrieb am Dienstag, 13. Juni 2023 um 15:38:53 UTC+2:
> Alexandru schrieb am Dienstag, 13. Juni 2023 um 15:32:13 UTC+2:
> > Schelte schrieb am Montag, 12. Juni 2023 um 21:27:20 UTC+2:
> > > On 12/06/2023 18:41, Alexandru wrote:
> > > > How where should I save your package?
> > > > I unpacked the ZIP archive to C:\Tcl\lib\www but package require www fails...
> > > As it is a Tcl module, you have to put it in one of the directories
> > > returned from [tcl::tm::path list], or add the directory where you put
> > > it to that list, using [tcl::tm::path add C:/Tcl/lib/www].
> > >
> > >
> > > Schelte.
> > Well, I'm a little bit familiar with the Tcl basics. I put your package as any other package into the recognized directory C:/Tcl/lib.
> > Should your archive have a pkgIndex.tcl ?
> > This file is not available so no wonder is not working.
> > I tried to define the pkgIndex.tcl with this content:
> >
> > package ifneeded www 2.3 [list source [file join $dir www-2.3.tm]]
> >
> > This give me the error:
> >
> > attempt to provide package www 2.3 failed: no version of package www provided
> Ah sorry, I think I know what the problem is. The tm are expected in another location than the normal libraries.
> Forget my last post.

Now I put the www directory directory into one of the paths returned by [tcl::tm::path list] and it can still not find the package.

On 13/06/2023 15:43, Alexandru wrote:
> Now I put the www directory directory into one of the paths returned by [tcl::tm::path list] and it can still not find the package.

The module system doesn't look in subdirectories. That was one of its
design goals: avoid delays due to scanning large portions of the file
system. So you should put the contents of the www directory in a path
returned by [tcl::tm::path list].

Alternatively, you can indicate that the package is in the www
subdirectory: package require www::www. That will work for this specific
case. But it will mess things up if one of the sub packages is needed,
such as www::digest, which should now be loaded as www::www::digest.

I didn't include an installation part in the documentation because it is
just a normal Tcl module. But if even a long-time Tcl user like yourself
is struggling with it, I guess I may have to add it.

Schelte.

Schelte schrieb am Dienstag, 13. Juni 2023 um 16:27:00 UTC+2:
> On 13/06/2023 15:43, Alexandru wrote:
> > Now I put the www directory directory into one of the paths returned by [tcl::tm::path list] and it can still not find the package.
> The module system doesn't look in subdirectories. That was one of its
> design goals: avoid delays due to scanning large portions of the file
> system. So you should put the contents of the www directory in a path
> returned by [tcl::tm::path list].
>
> Alternatively, you can indicate that the package is in the www
> subdirectory: package require www::www. That will work for this specific
> case. But it will mess things up if one of the sub packages is needed,
> such as www::digest, which should now be loaded as www::www::digest.
>
> I didn't include an installation part in the documentation because it is
> just a normal Tcl module. But if even a long-time Tcl user like yourself
> is struggling with it, I guess I may have to add it.
>
>
> Schelte.

Thanks, now it loads the package.
I'll test it.

Regards
Alexandru

Alexandru schrieb am Dienstag, 13. Juni 2023 um 16:54:51 UTC+2:
> Schelte schrieb am Dienstag, 13. Juni 2023 um 16:27:00 UTC+2:
> > On 13/06/2023 15:43, Alexandru wrote:
> > > Now I put the www directory directory into one of the paths returned by [tcl::tm::path list] and it can still not find the package.
> > The module system doesn't look in subdirectories. That was one of its
> > design goals: avoid delays due to scanning large portions of the file
> > system. So you should put the contents of the www directory in a path
> > returned by [tcl::tm::path list].
> >
> > Alternatively, you can indicate that the package is in the www
> > subdirectory: package require www::www. That will work for this specific
> > case. But it will mess things up if one of the sub packages is needed,
> > such as www::digest, which should now be loaded as www::www::digest.
> >
> > I didn't include an installation part in the documentation because it is
> > just a normal Tcl module. But if even a long-time Tcl user like yourself
> > is struggling with it, I guess I may have to add it.
> >
> >
> > Schelte.
> Thanks, now it loads the package.
> I'll test it.
>
> Regards
> Alexandru

When trying to open another URL (https://www.avi-gmbh.com/) your code throws and error that that aborts the my program flow, although I use catch:
*** START OF ERROR MESSAGE ***
SSL channel "sock0000000015597E60": error: sslv3 alert handshake failure
SSL channel "sock0000000015597E60": error: sslv3 alert handshake failure
*** END OF ERROR MESSAGE ***
*** START OF ERROR MESSAGE ***
software caused connection abort
software caused connection abort
while executing
"close $fd"
(class "::www::connection" method "Disconnect" line 6)
invoked from within
"my Disconnect"
(class "::www::connection" method "Failed" line 3)
invoked from within
"my Failed {WWW DATA TIMEOUT} "timeout waiting for a response""
(class "::www::connection" method "Timedout" line 2)
invoked from within
"::oo::Obj1007::my Timedout"
("after" script)
*** END OF ERROR MESSAGE ***

How can continue in case of an error?

On 13/06/2023 17:58, Alexandru wrote:
> When trying to open another URL (https://www.avi-gmbh.com/) your code throws and error that that aborts the my program flow, although I use catch

Thank you for this example of a failing scenario. I will investigate why
it is not handled correctly. But this may take some time.

Thanks,
Schelte.

Schelte schrieb am Mittwoch, 14. Juni 2023 um 15:41:07 UTC+2:
> On 13/06/2023 17:58, Alexandru wrote:
> > When trying to open another URL (https://www.avi-gmbh.com/) your code throws and error that that aborts the my program flow, although I use catch
> Thank you for this example of a failing scenario. I will investigate why
> it is not handled correctly. But this may take some time.
>
>
> Thanks,
> Schelte.

Hi Schelte,
Did you find time to look into this issue?
I have a task where I could use your package if it can catch communication errors.
Many thanks
Alexandru

On 19/07/2023 09:55, Alexandru wrote:
> Did you find time to look into this issue?
> I have a task where I could use your package if it can catch communication errors.

I did have a quick look. But in my simple test, it doesn't behave
consistent. Sometimes the error is caught. Other times it takes a while
before it is reported via the background error handler. That makes
debugging harder. And then I got distracted by other things :-}

But knowing there is a real need for this, increases the priority. I
will have another go at it.

Schelte.

Schelte schrieb am Mittwoch, 19. Juli 2023 um 10:47:00 UTC+2:
> On 19/07/2023 09:55, Alexandru wrote:
> > Did you find time to look into this issue?
> > I have a task where I could use your package if it can catch communication errors.
> I did have a quick look. But in my simple test, it doesn't behave
> consistent. Sometimes the error is caught. Other times it takes a while
> before it is reported via the background error handler. That makes
> debugging harder. And then I got distracted by other things :-}
>
> But knowing there is a real need for this, increases the priority. I
> will have another go at it.
>
>
> Schelte.

Yes, there is a real need. Thanks for the help.
Regards
Alexandru

On 7/19/2023 4:52 AM, Alexandru wrote:
>
> Yes, there is a real need. Thanks for the help.
> Regards
> Alexandru
>

Would you care to describe what you are doing with these websites?
Depending on your use, there may be other options.

Like: are you just downloading them to a local file? or checking
whether they are up and running? etc.

saitology9 schrieb am Mittwoch, 19. Juli 2023 um 20:54:13 UTC+2:
> On 7/19/2023 4:52 AM, Alexandru wrote:
> >
> > Yes, there is a real need. Thanks for the help.
> > Regards
> > Alexandru
> >
> Would you care to describe what you are doing with these websites?
> Depending on your use, there may be other options.
>
> Like: are you just downloading them to a local file? or checking
> whether they are up and running? etc.

Parsing the HTML content for specific keywords.
I already have the Tcl scripts ready for work.
The only 2 issues are described above (error catching or handling of cookies)

On 19/07/2023 09:55, Alexandru wrote:
> Did you find time to look into this issue?
> I have a task where I could use your package if it can catch communication errors.

I have found a problem and fixed it. In my testing, the latest committed
version of the www package now consistently allows catching the error to
access the URL you mentioned. Please let me know if you encounter any
more problems.

Thanks,
Schelte.

Schelte schrieb am Freitag, 21. Juli 2023 um 21:48:13 UTC+2:
> On 19/07/2023 09:55, Alexandru wrote:
> > Did you find time to look into this issue?
> > I have a task where I could use your package if it can catch communication errors.
> I have found a problem and fixed it. In my testing, the latest committed
> version of the www package now consistently allows catching the error to
> access the URL you mentioned. Please let me know if you encounter any
> more problems.
>
>
> Thanks,
> Schelte.
Thanks Schelte, I'll run the code with your updated package right away and give you feedback.

Schelte schrieb am Freitag, 21. Juli 2023 um 21:48:13 UTC+2:
> On 19/07/2023 09:55, Alexandru wrote:
> > Did you find time to look into this issue?
> > I have a task where I could use your package if it can catch communication errors.
> I have found a problem and fixed it. In my testing, the latest committed
> version of the www package now consistently allows catching the error to
> access the URL you mentioned. Please let me know if you encounter any
> more problems.
>
>
> Thanks,
> Schelte.
Your new code breaks earlier now at this link and with this error:

Getting https://globalautomationtechnologies.com/...(622/7537)
*** START OF ERROR MESSAGE ***
can't delete "::www::sock00000000154A58A0": command doesn't exist
can't delete "::www::sock00000000154A58A0": command doesn't exist
while executing
"rename ::www::$fd """
(class "::www::connection" method "Disconnect" line 5)
invoked from within
"my Disconnect"
(class "::www::connection" method "Failed" line 3)
invoked from within
"my Failed [dict get $opts -errorcode] $msg"
(class "::www::connection" method "Statusline" line 20)
invoked from within
"::oo::Obj756::my Statusline"
*** END OF ERROR MESSAGE ***

On 7/21/2023 4:11 PM, Alexandru wrote:
> Schelte schrieb am Freitag, 21. Juli 2023 um 21:48:13 UTC+2:
>> I have found a problem and fixed it. In my testing, the latest committed
>> version of the www package now consistently allows catching the error to
>> access the URL you mentioned. Please let me know if you encounter any
>> more problems.
>>
>>
>> Thanks,
>> Schelte.
> Your new code breaks earlier now at this link and with this error:
>
> Getting https://globalautomationtechnologies.com/...(622/7537)

I noticed that whenever you get these errors, Firefox and Chrome also
complain. It is because these sites do not have their certificates setup
but the url's start with https. If you use http instead in the same
url's, it works and redirect works.

saitology9 schrieb am Freitag, 21. Juli 2023 um 23:28:42 UTC+2:
> On 7/21/2023 4:11 PM, Alexandru wrote:
> > Schelte schrieb am Freitag, 21. Juli 2023 um 21:48:13 UTC+2:
> >> I have found a problem and fixed it. In my testing, the latest committed
> >> version of the www package now consistently allows catching the error to
> >> access the URL you mentioned. Please let me know if you encounter any
> >> more problems.
> >>
> >>
> >> Thanks,
> >> Schelte.
> > Your new code breaks earlier now at this link and with this error:
> >
> > Getting https://globalautomationtechnologies.com/...(622/7537)
> I noticed that whenever you get these errors, Firefox and Chrome also
> complain. It is because these sites do not have their certificates setup
> but the url's start with https. If you use http instead in the same
> url's, it works and redirect works.
Thanks for the hint.

Still, changing the URL from https to https won't solve the problem, since other sites will fail because of http.
Parsing 7000 sites need an error catching system.