Re: status of lily server changes?

Christian A Ratliff (ratlifc@biddeford.com)
Fri, 19 Jan 1996 21:24:06 -0500

On Fri, 19 Jan 96 20:08:00 -0500 Garance A Drosehn wrote:
>
>> 1] Use the embryonic $transfer code to log state to a file rather
>> than checkpointing the database. This would eliminate the need
>> to fork() and give us a portable copy of the database for
>> upgrades as well as checkpoints.
>>
>
>The server is slowed down during checkpoints as it is. This might
>be a worthwhile direction, *if* we can be sure to get a consistent
>set of objects. An inconsistent checkpoint isn't worth a whole lot.
>Of course, it also depends on just how much this checkpoint method
>really would slow down the server...
>
Yeah. The ideas you put forward later in the letter make this not
quite as significant.

>> 2] Don't fork() during checkpointing and checkpoint once a day.
>> This stops the growing swapfile problem.
>
>Boo, hiss. Negatory.
>
Good. :)

>> 3] Rewrite the malloc() calls in MOO to use NeXTstep vm_* functions.
>> By holding onto the vm_allocate() returns perhaps we can use
>> MACH's builtin love for memory-based IPC to implement a method
>> of forking which is not actually forking and does not touch off
>> the bug.
>
>I am not optimistic about the chances of this working out well.
>
Yeah. An idea, not a great one.

>> It might help to know more about the nature of the bug though.
>
>I have a suspicion that it would solve the ever-growing swapfile
>problem to have it:
> read thru process memory
> write all the objects to disk
> loop thru all the objects again
> *modifying* memory (thus causing a copy-on-write),
> and then freeing that object.
> once done with all objects, exit the process
>
This is definately possible. I will look into this and give you an idea
of the possiblity. Also, I think there is a way (a NeXTstep vm_() call)
to tell it not to C-o-W pages. I will check that out too.

>One thing to note is that it really can help a lot just to have
>less information kept around.
>
One really big way to reduce this is to pre-supply a single message
output style (which I call the 'ideal' message). Every time you type in
'foo; bar' each user in 'foo' gets their own formatted copy of the message.
I think it would be a good idea to make one 'ideal copy as a shared
string before the calls to all of the player message delivery functions.
Then if you can see all recipients and have an 80 column screen you use
the 'ideal' (shared) copy. This will probably cut the number of individal
strings by a fairly high degree.

>I really think that having events expire out of discussions will
>also reduce the rate of growth.
>
I actually think this is a fairly reasonable fix. I will explore it and
report back.

>In december my fear was that by the end of January we'd be in a
>crisis situation, requiring some significant money to be spent.
>"Crisis situation", to me, means having a reboot frequency of less
>than seven days. Now it looks like we'll do OK for a bit longer,
>and as memory prices drop (and they actually have dropped a bit)
>the amount of money to spend isn't as bad. If we can get thru
>this semester OK, that would be nice (it'd let my bank account
>recover, or at least I hope it will...).
>
There are many things which can be done to relieve this pressure at a
code level. I will see what I can do on this side to deal with it.

christian

-----
Christian A. Ratliff / Chief Technical Officer / techs@biddeford.com
(NeXTmail & MIME okay) / Designer and Co-Author / ratlifc@lily.org
http://www.biddeford.com/~ratlifc/