On Thu, 13 May 2021 16:11:32 +0100
James Harris <james.harris.1@gmail.com> wrote:

> On 13/05/2021 09:35, Rod Pemberton wrote:
> > On Sun, 9 May 2021 06:32:29 -0700 (PDT)
> > James Harris <james.harris.1@gmail.com> wrote:

> >> OK, but how will the code know where its statically declared
> >> variables are in memory? In the example above /how/ will the code
> >> know to look at address 0x400 for one instance and 0x420 for
> >> another? That's the problem I've been trying to solve.
> >
> > I don't understand why two GOTs are so close together.
>
> Sorry, I was just keeping the numbers small to illustrate the
> situation. More realistically, the code could be at 0x0A05_1000 and
> there could be two executing instances, one with its data at
> 0x0100_4000 and another with its data at 0x01B0_0000, say.
>

Ok. One copy of code, being executed multiple times. Multiple data
areas (or GOTs etc).

How do you determine which "instance" of the code is executing at any
given time? I.e., if you're executing the code twice, what returns #1
for the first instance and number #2 for the second instance. Can't
this be used to look up the address of each data area (or GOT)?

--
"There's the abyss. We're not in the abyss. We're in the car park and
snack area adjacent to the abyss." - Bill Bailey

On 16/05/2021 07:35, Rod Pemberton wrote:
> On Thu, 13 May 2021 16:11:32 +0100
> James Harris <james.harris.1@gmail.com> wrote:
>
>> On 13/05/2021 09:35, Rod Pemberton wrote:
>>> On Sun, 9 May 2021 06:32:29 -0700 (PDT)
>>> James Harris <james.harris.1@gmail.com> wrote:
>
>>>> OK, but how will the code know where its statically declared
>>>> variables are in memory? In the example above /how/ will the code
>>>> know to look at address 0x400 for one instance and 0x420 for
>>>> another? That's the problem I've been trying to solve.
>>>
>>> I don't understand why two GOTs are so close together.
>>
>> Sorry, I was just keeping the numbers small to illustrate the
>> situation. More realistically, the code could be at 0x0A05_1000 and
>> there could be two executing instances, one with its data at
>> 0x0100_4000 and another with its data at 0x01B0_0000, say.
>>
>
> Ok. One copy of code, being executed multiple times. Multiple data
> areas (or GOTs etc).
>
> How do you determine which "instance" of the code is executing at any
> given time? I.e., if you're executing the code twice, what returns #1
> for the first instance and number #2 for the second instance. Can't
> this be used to look up the address of each data area (or GOT)?

AFICS the simplest thing would be to pass to each piece of code a
pointer to its data. In the example above, the single piece of code
would be passed a pointer to 0x0100_4000 or 0x01B0_0000 as required.

BTW, here's a list of potential advantages as I see them.

* Never any need to fix up code to make it executable. Just 'load and go'

* In each address space a piece of code could be placed anywhere. The
location could be chosen to suit the task rather than being best placed
in a location prebuilt into the executable image

* No possibility for location conflict between loadable modules

* No need for a registry of preferred locations for library modules, and
no need to leave space between preferred locations for future expansion

* No chance that the preferred locations of one set of libraries will
conflict with some other set from a different software house

* No chance that the locations of library modules will begin to conflict
as code is upgraded

* No need to set aside a large range of addresses against the
possibility that certain libraries will be loaded. The entire address
space would be available to the programmer and there would be nothing in
an awkward place to prevent allocating memory for a very large object,
if desired. (64-bit address spaces are so large they would not have this
problem)

* Option to use paging hardware, where present, to map code from disk to
memory rather than to loading it explicitly

* Unencumbered sharing of executable pages between address spaces
without any of them needing to be altered. Further, if an executable
page is already in memory or is even mapped to memory for another task
then it could be used as it stands

* Flexibility to run each instance of a task in its own address space or
to let multiple instances share an address space depending on OS
workload and a user's goals

* Option to move code - perhaps more than once, e.g. for compaction -
after it has been loaded

* Freedom to burn the code to ROM unchanged after testing without
knowing in advance which addresses the ROM will occupy

* No changes necessary on systems which, for security and other
purposes, implement Address Space Layout Randomization

Of course, it has disadvantages.

* Need to pass one or more pointers to impure functions. (Pure functions
would be the same in either case)

* Cost of passing those pointers

* Additional cost of accessing global data indirectly, especially where
the set of available registers is small

Basically, aside from its parameters a function would have access to

1. internal state which lasts just while the function is executing
2. external state which will persist between invocations

and no other data.

Pure functions wouldn't need type 2.

Impure functions /would/ need type 2 and would be told where to find it
via one or more pointers.

I suppose the million dollar question is whether this is a good idea? I
don't know. It's certainly an intriguing model and hints at unusual ways
to compose systems.

--
James Harris

On Sun, 16 May 2021 19:59:01 +0100
James Harris <james.harris.1@gmail.com> wrote:

> On 16/05/2021 07:35, Rod Pemberton wrote:
> > On Thu, 13 May 2021 16:11:32 +0100
> > James Harris <james.harris.1@gmail.com> wrote:
> >> On 13/05/2021 09:35, Rod Pemberton wrote:
> >>> On Sun, 9 May 2021 06:32:29 -0700 (PDT)
> >>> James Harris <james.harris.1@gmail.com> wrote:

> >>>> OK, but how will the code know where its statically declared
> >>>> variables are in memory? In the example above /how/ will the code
> >>>> know to look at address 0x400 for one instance and 0x420 for
> >>>> another? That's the problem I've been trying to solve.
> >>>
> >>> I don't understand why two GOTs are so close together.
> >>
> >> Sorry, I was just keeping the numbers small to illustrate the
> >> situation. More realistically, the code could be at 0x0A05_1000 and
> >> there could be two executing instances, one with its data at
> >> 0x0100_4000 and another with its data at 0x01B0_0000, say.
> >>
> >
> > Ok. One copy of code, being executed multiple times. Multiple data
> > areas (or GOTs etc).
> >
> > How do you determine which "instance" of the code is executing at
> > any given time? I.e., if you're executing the code twice, what
> > returns #1 for the first instance and number #2 for the second
> > instance. Can't this be used to look up the address of each data
> > area (or GOT)?
>
> AFICS the simplest thing would be to pass to each piece of code a
> pointer to its data.

At the program level? Ok.

At the procedure level? Um, ... Big look-up table?

I'm thinking it makes sense at the program level. Your OS is probably
keeping track of some system FILE pointers, and stacks for each app
already.

> BTW, here's a list of potential advantages as I see them.
>
> * Never any need to fix up code to make it executable. Just 'load and
> go'
>
> * In each address space a piece of code could be placed anywhere. The
> location could be chosen to suit the task rather than being best
> placed in a location prebuilt into the executable image
>
> * No possibility for location conflict between loadable modules
>
> * No need for a registry of preferred locations for library modules,
> and no need to leave space between preferred locations for future
> expansion
>
> * No chance that the preferred locations of one set of libraries will
> conflict with some other set from a different software house
>
> * No chance that the locations of library modules will begin to
> conflict as code is upgraded
>
> * No need to set aside a large range of addresses against the
> possibility that certain libraries will be loaded. The entire address
> space would be available to the programmer and there would be nothing
> in an awkward place to prevent allocating memory for a very large
> object, if desired. (64-bit address spaces are so large they would
> not have this problem)
>
> * Option to use paging hardware, where present, to map code from disk
> to memory rather than to loading it explicitly
>
> * Unencumbered sharing of executable pages between address spaces
> without any of them needing to be altered. Further, if an executable
> page is already in memory or is even mapped to memory for another
> task then it could be used as it stands
>
> * Flexibility to run each instance of a task in its own address space
> or to let multiple instances share an address space depending on OS
> workload and a user's goals
>
> * Option to move code - perhaps more than once, e.g. for compaction -
> after it has been loaded
>
> * Freedom to burn the code to ROM unchanged after testing without
> knowing in advance which addresses the ROM will occupy
>
> * No changes necessary on systems which, for security and other
> purposes, implement Address Space Layout Randomization
>
>
> Of course, it has disadvantages.
>
> * Need to pass one or more pointers to impure functions. (Pure
> functions would be the same in either case)
>
> * Cost of passing those pointers
>
> * Additional cost of accessing global data indirectly, especially
> where the set of available registers is small
>

If you keep this to a program level, and not a procedure level, then I
don't think the "Cost of passing those pointers" is all that much.

With a list that long of advantages, you wouldn't be dissuaded by
anything I said, even if I had something to counter your position,
which I don't, currently.

--
The SALT deduction is a kickback of taxes to wealthy people in wealthy
states.

On 24/05/2021 03:28, Rod Pemberton wrote:
> On Sun, 16 May 2021 19:59:01 +0100
> James Harris <james.harris.1@gmail.com> wrote:

....

>> AFICS the simplest thing would be to pass to each piece of code a
>> pointer to its data.
>
> At the program level? Ok.
>
> At the procedure level? Um, ... Big look-up table?

I've been giving this some more thought. I think the answer depends on a
number of factors: whether the program is to run in its own address
space, whether it is to share address space with other programs, what
security is required between programs, whether the architecture can use
segmentation or paging to achieve separation or has no provision
therefor, etc, along with how to link programs together and how to allow
different versions of the same program to coexist (such as having an
older version and an upgrade running at the same time).

Ideally, compilers and assemblers would produce object code which could
be used in any of those scenarios. The only difference would be in how
the pieces of object code were linked together - e.g. how one program
would call another.

I've long thought it best to make the unit of program distribution
/object files/ rather than executable files, linking them together as
needed, and this seems to be another case where that would be more
flexible than having linker and loader separate.

The trouble is it suggests writing my own linker, and I've already got
enough to do.... :-((

That said, for my OS project I will need some kind of loader. Perhaps it
would be easier in the long term just to get it to do the linking on
demand.

>
> I'm thinking it makes sense at the program level. Your OS is probably
> keeping track of some system FILE pointers, and stacks for each app
> already.

....

>
> If you keep this to a program level, and not a procedure level, then I
> don't think the "Cost of passing those pointers" is all that much.
>
> With a list that long of advantages, you wouldn't be dissuaded by
> anything I said, even if I had something to counter your position,
> which I don't, currently.

I'm even having second thoughts about the execution cost: maybe it's not
as high as I thought. AIUI languages which allow a programmer to have
one function embedded in another already pass a pointer from the outer
function to the inner one so it can access variables in the outer
environment. (Essentially, this is a pointer to the environment of
definition.) For example, consider

function A
var X
function B
print X
endfunction B
B()
endfunction A

For B to have access to X the call B() would pass a hidden pointer to
A's variables. (C, by contrast, doesn't allow function nesting.)

If that's right then there are two common areas in which pointers are
passed anyway:

1. Pointers to objects are passed to OOP methods.

2. Pointers to outer environments are passed to inner functions.

They are both very common and people accept them without question. So
maybe I shouldn't worry too much about the cost of such facilities to
produce PIC code along with its advantages.

--
James Harris

On 28/05/2021 12:28, James Harris wrote:
> [...] AIUI languages which allow a programmer to
> have one function embedded in another already pass a pointer from the
> outer function to the inner one so it can access variables in the
> outer environment. (Essentially, this is a pointer to the environment
> of definition.) For example, consider
>   function A
>      var X
>      function B
>         print X
>      endfunction B
>      B()
>   endfunction A

There's no need to pass a pointer. "X" is accessed inside
"B" in exactly the same way it is accessed inside "A"; it's 27
bytes [or whatever] up from "A"'s stack base, similar to anything
else declared locally to "A". In general, you therefore need access
to all currently-accessible stack bases; this is commonly done by
having a small array [of size the maximum static nesting depth, two
in your example] of stack bases which is maintained on procedure
entrance/exit. You need this anyway if functions can be recursive.
Other techniques are possible, and may be necessary in the presence
of threads or parallel processing or similar.

> For B to have access to X the call B() would pass a hidden pointer to
> A's variables. (C, by contrast, doesn't allow function nesting.)

So much the worse for C; it was almost a killer when we
first encountered C. We learned to live with it, but it has always
been a nuisance, esp when translating programs from other languages.
[Likewise static arrays, but there were always minimally-acceptable
ways around that.]

--
Andy Walker, Nottingham.
Andy's music pages: www.cuboid.me.uk/andy/Music
Composer of the day: www.cuboid.me.uk/andy/Music/Composers/Vivaldi