[ros-kernel] RE: I can boot ReactOS with 8 Mb!!

Sat Feb 7 08:13:29 CET 2004

Hi,

I've looked to add_to_free_list. There is a bug for one condition which
will corrupt the non paged pool. If a block is freed and the previous is
also free and the next not, the previous pointer from the next block
isn't set. This does corrupt the non paged pool at that time when the
the next block is freed. After some testing, I will commit a fix. 

- Hartmut

-----Original Message-----
From: Waldo Alvarez Cañizares [mailto:ros-kernel-bounces at reactos.com] On
Behalf Of Waldo Alvarez Cañizares
Sent: Tuesday, February 03, 2004 7:02 PM
To: ReactOS Kernel List
Subject: RE: [ros-kernel] RE: I can boot ReactOS with 8 Mb!!

Hi Hartmut:

I'm sure now there is a bug in there. I took the original code and
enabled DUMP_AVL I got a node sowing size 0 late on the boot process, it
later vanished but was there for quite long time. add_to_free_list seems
correct. The most I remember is that I was tracing that code in add_to
_free_list and got a block not marked as used or as free. That was after
add_to_free... was called by get_block. If you want I can send you the
code without the bug. The problem is that o don't remember exactly where
it was and I do not like the idea of refinxing what I already have
fixed.

Best Regards
Waldo Alvarez

From: ros-kernel-bounces at reactos.com on behalf of Waldo Alvarez
Cañizares
Sent: Thu 1/29/2004 4:52 PM
To: ReactOS Kernel List
Subject: RE: [ros-kernel] RE: I can boot ReactOS with 8 Mb!!

Hello Hartmut:

From: ros-kernel-bounces at reactos.com on behalf of Hartmut Birr
Sent: Thu 1/29/2004 1:46 PM
To: 'ReactOS Kernel List'
Subject: [ros-kernel] RE: I can boot ReactOS with 8 Mb!!

> -----Original Message-----
> From: Waldo Alvarez Cañizares
> [mailto:ros-kernel-bounces at reactos.com] On
> Behalf Of Waldo Alvarez Cañizares
> Sent: Thursday, January 29, 2004 12:22 AM
> To: ReactOS Kernel List
> Subject: I can boot ReactOS with 8 Mb!!
>
>
> that posible? There is a bug in the nonpaged pool where when
> a block of
> memory is freed, the previous adjacent to is not coalesced,
> thus causing
> checkerboarding, fixing it is trivial but I have almost ready a patch
> that redesigns all things in there(I'm going to add that check to the
> macro VALIDATEPOOL to avoid it from happening again). I took

If there is a bug in the non paged pool, I would like to fix this bug
without other changes.
I saw that one with 0.1.5 I have a bad memory and I'm not at home. If  I
remember well I saw it in add_to_free_list I have not taken a look at
recent releases. I downloaded 0.2 about 2 days ago. when you free a
block and there is a previous block free it does not joins those blocks.
However check it too if you want, I'could be wrong about that. I'll
recheck the original implementation and if it is for real I will quote
the piece of code here.

> away memory
> blocks headers out of the nonpaged pool and putted them in a
> stack that
> spans across a couple of pages avoiding address translations while
> searching over the tree across the block all over the memory.
> I removed
> those extra memset's, and changed the AVL tree by a Red-Black Tree.

Some time ago, I've implemented the AVL tree. I haven't written the real
AVL code. It  was easier to found examples for an AVL tree as for an
other balanced tree. If the Red-Black trees are faster or it is easier
to understand the code, we should change to Red-Black trees.
Actually the code is harder to understand, but they are faster inserting
and deleting and slower searching because are semibalanced trees, they
can have a depth 2 times of he avl but they rotatate a lot less. Half of
the times they don't rotate and at most rotate 2 times when deleting a
node. I also took already implemented code but I modified it to reduce
its size and increase its speed and took some of the avl functions
(those to search) from ROS. I'm thinking that I could also implement the
memory management there using some sort of buddy system. That is an
array of pointer to nodes in the tree sorted in ranges of sizes to
reduce the search time but i'm still not sure on how to implement it, I
want to finish some things before doing that like using a circular list
for the list that have blocks sorted by address an avoid to extract and
remove a node from the tree if it can stay at that point after it's size
changes. I was planning to implement it to move the node if it can stay
there but that is a little more complicated than extracting it and
insert it later. I have to fix the allocations of aligned memory. Also i
want to put a coockie in the TAG to check if a block of memory is
overwriten since I took away the block headers from there.

> However, there are a couple of things. One I need to know the
> memory map
> of the reactos executive because I need a free available block to put

You can look into MmInitVirtualMemory. There is the basic initialisation
of reserved memory areas.

c0000000-zzzzzzzz ntoskrnl and boot drivers
zzzzzzzz-yyyyyyyy ntoskrnl init section
yyyyyyyy-xxxxxxxx ntoskrnl bss section
xxxxxxxx-wwwwwwww boot parameter area
wwwwwwww-vvvvvvvv non paged pool
vvvvvvvv-uuuuuuuu kernel map
uuuuuuuu-tttttttt paged pool
d0000000-d00fffff low memory mapped into kernel space
f0000000-f03f0000 page directories for a process
ff000000-ff01ffff kpcr's for 32 processors
ffdf0000-ffdf0fff shared user page mapped into kernel space
Thanks, thanks a lot. that means that after tttttttt i can put something
if it does not overlaps with the cache manager.

All other ranges are used by the cache manger or other function which
does reserve memory.
I know that one. If i remember it's size is set to 100Mb as with the
nonpaged pool.

> physical page to later map it) and faced that the reactos
> cache manager
> uses the non paged pool (wich is locked at that moment, and
> that I can't

You must always look the access to the nonpaged pool structures. If the
access is not locked, a thread switching (or an other thread on a smp
machine) can result in a second call to the pool functions.
I know. That is why i can't make the cache manager to trim its size.

> unlock or I'll cause race conditions) I tested it unlocking
> the nonpaged
> pool so that the memory from the cache manager could be freed and I

The cache manger uses only the nonpaged pool for internal datas. The
cache segments are allocate via the memory areas.
Yes but when I tell it to Trim its size it does access the nonpaged
pool. That means while trying to allocate a page the function that maps
pages returns that there are not available pages when the cache manager
is using about 2 Mb wich mean that there is plenty of room to allocate
the pages required by the nonpage pool. I saw also that when you tell
the cache manager to trim too much, it crashes with apage fault when it
is using about 40 pages, so it seems that there is some buggy code.

> And finally I'm getting calls to allocate 8, 16, 23, 72 ..
> and such tiny
> quantities of memory. Hmm that's really bad actually a such a call can
> allocate up to one page which is a looot more than what you
> ask. I think

For some datas is it possible to allocate them from the stack. But some,
like rmap entries, must be always allocate from the pool.
Sure I'm aware that some data even small ones can't, but we can do
something about those that can.
Remember also that >>sometimes<< you can allocate a block of memory at
once to put many of those small structures instead of asking them one by
one.

- Hartmut
Best reegards
Waldo Alvarez

_______________________________________________
Ros-kernel mailing list
Ros-kernel at reactos.com
http://reactos.com/mailman/listinfo/ros-kernel