[ros-dev] Re: proper nt-style authentication (reactos, wine, samba tng)

[David Johnson]

I love for this great idea and I put it to the ROS DEV..... 
Please consider doing every bit of it.

XXXOOO

Dave

On 9/1/05, Luke Kenneth Casson Leighton <lkcl at lkcl.net> wrote:
> [please note: due to its cross-discipline and cross-project nature,
>  this message is going out to SEVERAL related project mailing lists.
>  when replying, please take EXTRA caution not least because some of
>  these lists are subscriber-only.  also, please note: i _would_
>  post this also to the samba mailing lists but due to the fascist
>  censorship in place since the 15th dec 2004, i am unable to do so.
>  this is their decision and it is their loss.  i am not asking you
>  to respect that decision i am simply making you aware of it.]
> 
> hi,
> 
> out the woodwork i pop - not necessarily ready to chew anything because
> i know just how much work's involved, but what i did want to do was
> say "hi, i'm still here" and do a brain-dump of how authentication
> ideally needs to be implemented in reactos.
> 
> at 2,600 words and 16k, this message is quite long and so i have
> placed a copy at http://lkcl.net/software/reports/reactos-auth.txt,
> just in case it doesn't make it past various mailing-list limits
> (i'll find out in a couple of minutes... :)
> 
> it breaks down into a number of sections:
> 
> 	1) i describe the timescales and ways to cut them,
> 	   along with some warnings and stuff.
> 
> 	2) amonst other stuff i outline some background as to
> 	   why i am posting this to so many lists.
> 
> 	3) i outline a project plan and the dependencies
> 	   and "optional" steps,
> 
> 	4) i describe a recommended implementation
> 	   roadmap starting with the "minimum" requirements,
> 	   and expand on some technical info and references
> 	   i found, which would help with some of the
> 	   "nice-to-have" steps.
> 
> so.  first.
> 
> please do not be scared by how much work is involved, and how much code
> there is.  it all hinges one ONE function and that one function... you
> are _not_ going to believe how much code that one function drags in,
> kicking and screaming, behind it.
> 
> please also please i beg you DO NOT consider going "bleedin 'ellfire,
> that's so much frakkin work we can't POSSIBLY do it the way you
> describe, we MUST do it our own way, starting with what we know, love
> and have already started tackling, and can't possibly back out of what
> we've already done".
> 
> should you choose to exercise the "NIH" option, i frakkin
> guarantee you that you will waste about five to eight years
> of your (collective) lives reinventing the wheel.
> 
> "The Plan" outlined here will shave that down to about 12 to 18 months,
> utilising some SIX HUNDRED THOUSAND lines of pre-existing code, and
> later in this message i also outline a prioritisation of the necessary
> work to "cut down" the time to maybe about ...  mmm... three or so 
> months, by leaving out some of the "nice-to-have" stuff.  actually...
> _all_ of the nice-to-have stuff :)
> 
> that will at least "GetItWorking(tm)" and the rest of the bits
> can be considered at leisure once people go "god that's awful,
> we can't possibly leave it like that" and hopefully hopefully
> things will actually progress from there.
> 
> remember - please: this stuff is sufficiently complicated such that
> you really can't afford the niceties such as "It Must Be Perfect (tm)"
> before it can be accepted.  i've seen that shit before, and it's
> a baad mxxxxxxxer path to go down, especially with such complex
> and heavily interdependant reverse-engineering projects as reactos,
> wine, samba and freedce.
> 
> anyway.
> 
> fyi - before i begin, i should mention a couple of things: 
> 
> 1) this message also goes out the the apache devel mailing
> list (specifically the APR one) because the NT style
> authentication thing is the sort of thing that really _should_
> be in a (special?) APR util library, along with "NT Named Pipes" -
> see http://en.wikipedia.org/wiki/NamedPipes
> 
> one of the reasons why NT-style NamedPipes is _not_ in an APR util is
> because it is believed that NT-style NamedPipes do not fit the
> "least common denominator".  by having the infrastructure outlined
> below, it is possible to move things "up one level" to the point
> where unix _has_ that denominononmination.
> 
> also, if APR still has support for that god-awful program-running
> program called Win95, it would, with not much extra effort, be
> possible to port some reactos components to Win95 (!) such that
> ThePlan outlined here would make Win95 have proper NT-style
> authentication (!) now there's a horrible thought that will give
> MS and free software people nightmares both: upgrading Win95 by
> installing free software components at it.  muhahahahah ahem.
> 
> 2) i have filled in a number of pages on wikipedia.org.  see
> http://en.wikipedia.org/wiki/User:lkcl.  please take the time to
> review these pages, PLEASE help me with my totally biased "POV"
> (point-of-view) comments, by either editing the pages direct or by adding
> comments on the "talk" pages as appropriate.  or alternatively dumping
> me in the nearest pond if ever you meet me.
> 
> wikipedia is supposed to be encyclopedia-ic and some of my
> comments are anything but that.
> 
> help!!!
> 
> 3) please due to the quite broad distribution of this message across
> multiple mailing lists, please do NOT ask stupid questions like
> "what the hell do we need to use samba tng for, why can't we use samba
> in reactos"? and "why is this guy bothering us with this insane stuff?"
> and "how do i use killfiles?" or terry pratchett's - more specifically
> the librarian's favourite - "where has all the pie and custard gone?"
> 
> please do your research: see if nothing else
> http://en.wikipedia.org/wiki/Samba_TNG_software and then start googling.
> 
> 
> anyway.
> 
> onward.
> 
> roughly in order of dependence, with "nice-to-have" status added as
> well:
> 
> 1) port FreeDCE to Win32 - more specifically add in autoconf support for an
> alternative threading model - the one in reactos.  no, this is not a joke:
> MSRPC is a critical part of NT infrastructure it's just that very few
> people
> are actually aware of this (including the US DoJ and the EU commission...)
> 
> see http://www.google.co.uk/search?q=lkcl+wine+freedce for discussion
> links as to why - it's a long story and it took me several dozen
> messages spanning over many days to outline it in sufficient detail
> for it to be understood (mostly me getting things straight in my
> head...)
> 
> status: "nice to have" but you will soon wish that you _did_ have it!!!
> 
> there is also an opportunity to support Wine, here: essentially
> it's the same job, for DCOM, for proper "authentication"
> purposes in Wine just like in ReactOS... it's a long story.
> 
> 2) remove dependence on "unix" security model from Samba TNG's services
> starting with samrd (perhaps by finishing off samrtdbd?)
> 
> status: "essential".
> 
> sub-projects:
> 
> a) in Samba TNG, rework winbindd such that it is a REQUIRED service,
>    and it has "modes" that, instead of "inventing" links between
>    unix uids and nt sids, it reads smbpasswd and correlates unix uids
>    with nt sids _just_ like is done now - but hard-coded through
>    the awful code that has had to dieee since... about two weeks after
>    i wrote it, back in 1999.
> 
>    this one's a long story but it's not actually required for
>    reactos but _might_ be required for wine...
> 
> status: "pretty much essential" but not for reactos but for making life
> easy to share tng development between reactos and unix.
> 
> b) in Samba TNG, make the authentication code (smb-side and rpc server
>    back-end-wise) contact winbindd BUT ONLY for resolving unix uids and
>    unix gids - which is actually a VERY small task involving a few
>    hundred lines of code.
> 
>    this goes back to the "SURS" stuff i wrote up in 1999 - but instead
>    of a library it hands off the responsibility for sid<->uid+gid
>    lookups to winbindd.
> 
>    it _may_ also be essential for Wine to operate correctly
>    with authentication (which is a "stub" at the moment just
>    like it is in reactos), to interoperate with other programs,
>    to cache login credentials in order to do smb client-side
>    operations, that sort of thing, under which circumstances,
>    it will be necessary for Wine to have the same sort of thing.
> 
>    it's no big deal :)
> 
> status: same as for 2a).
> 
> c) in Samba TNG, abandon all use of hard-coded MSRPC stuff and utilise
>    FreeDCE instead, and do a rewrite of all services, one by one
>    (and because samba tng is modular, this _can_ be done one-by-one,
>     checking MSRPC interoperability along the way between the "old"
>     services, the "new" services, and also with NT itself).
>    
>    in and of itself, this critically depends on making
>    DCEThreads reliable, though, or adding support (finally!) for
>    POSIX threads.
> 
> status: "non-essential" but you would soon end up wishing that you had :)
> 
> 3) complete LsaLogonUser, add LsaLogonUserEx, in NTOSKRNL.DLL, and
>    friends.
> 
>    my favourites.  the LSASS stuff.
> 
>    these functions are quite simple: they are "redirectors" - a vector
>    table of functions is required to be passed in, which includes things
>    like "authenticate with my lovely service" and "free some memory".
> 
>    there are many LSASS sub-services: one of them is Netware, one of
>    them is MSV1_0.DLL, one of them is Kerberos.
> 
>    there are others.
> 
>    LSASS is based on _exactly_ the same technique that dlopen does e.g
>    in libdvdcss, and in freedce's ntlmsspauth.so, and in... mmm...
>    the freedce transport module infrastructure and _many_ other
>    projects ...  except of course it's NT-based table of
>    higher-order-functions not unix "dlopen()" ones.
> 
>    i'll stop trying to teach people to suck eggs, now *embarrassed* :)
> 
> status: "essential".
> 
> sub-projects:
> 
> a) write an MSV1_0.DLL which registers with the LSASS service.
>    this will utilise MSRPC functions that call into the Samba TNG
>    "NETLOGON" service.
> 
>    there already exists "basic" functions inside Samba TNG that...
>    well... it's a bit messy, but they work.
> 
>    the key function to be calling is cli_nt_login_interactive,
>    and you will notice _very_ quickly from the arguments hey, some
>    of those look... kinda... familiar!!
> 
> see
> http://viewcvs.samba-tng.org/cgi-bin/viewcvs.cgi/tng/source/rpc_client/cli_netlogon.c
> 
> status: "essential."
> 
> the "porting" bit of this code to FreeDCE i would classify as
> "non-essential" but again, you will soon wish that you _were_ using
> FreeDCE.
> 
> 
> 
> that's basically it: there are other sub-projects such as
> turning the "Registry" code in ReactOS (or Wine) into an MSRPC
> service (yes, i did say pretty much everything that's anything
> critical in NT is an MSRPC service, didn't i? :) but they aren't
> "essential".
> 
> i say basically it, but that ONE stupid function, cli_nt_login_interactive,
> drags in quite literally HALF A MILLION lines of code - 250,000 lines of
> it in FreeDCE, alone (which, like i said, could possibly be avoided but
> the hard-coded MSRPC stuff in samba tng is sufficiently awkward to make
> a rewrite utilising FreeDCE very attractive).
> 
> 
> 
> what i would suggest is the following:
> 
> 1) write an MSV1_0.DLL "test stub" in combination with completing the
> LSASS functions, which supports a username "test", domain of "test" and
> password of "test".
> 
> code up a hard-coded "blob" - a NET_USER_INFO_3 structure - that contains
> the response / essential information of groups, gids, sids, user session
> key etc.
> 
> see
> http://viewcvs.samba-tng.org/cgi-bin/viewcvs.cgi/tng/source/include/rpc_netlogon.h
> 
> (note: i _did_ say it would be nice to utilise FreeDCE didn't i?  well,
> the NET_USER_INFO_3 structure in that header file is a "messy" version that
> i recreated from off-the-wire back in about ... mmmm... 1997.  there
> _do_ exist IDL-generated versions of this data structure, thanks to
> matthew chapman - NETLOGON.idl for example).
> 
> once that works...
> 
> 2) write a lovely insecure method of "outsourcing" the username,
> domain and password to an external server - Samba TNG - which performs
> the authentication on your behalf and gets back "real" data.
> 
> this could be done simply with a TCP connection, throw the data
> in-the-clear over to a simple temporary shim service blah blah,
> bob's your uncle.
> 
> 3) port samba tng's netlogond, samrd and lsarpcd to ReactOS.
> 
>    this is quite straightforward: about the only really essential
>    "missing" bit is to tie the services in to "NT Named Pipes" rather
>    than unix domain sockets.
> 
>    _but_ - they only need to be plugged in to a back-end transport
>    which all services - any of samba tng's MSRPC services - would
>    use.  so there's a key bit of work needed - probably under
>    400 lines of code - and the rest should fall into place.
> 
>    this is where it would be SO much easier to be utilising FreeDCE.
>    
>    all that's needed would be to write a FreeDCE "transport"
>    plugin for ReactOS - ncacn_np - and then you'd be DONE.
> 
>    it's a long story...
> 
>    hey, has someone implemented TransactNamedPipe() and CreateNamedPipe()
>    in ReactOS?
> 
>    if not, it's quite straightforward to do, and it involves dropping
>    opaque data blobs onto smbd (again, smbd ported to reactos...)
> 
>    i _did_ say it's a long story, didn't i? :)
> 
> 4) finally: track down how "LsaLogonUserEx" works.  LsaLogonUser
> utilises cleartext passwords.  LsaLogonUserEx i BELIEVE utilises NT and
> LM password hashes, or some brain-dead encrypted variant thereof.
> 
> once 1) is completed, then TA-DAAAAA!!! lib/secur/lsa.c's "LsaLogonUser"
> function actually gets REAL DATA!
> 
> 
> okay.
> 
> notes:
> 
> 1) regarding FreeDCE.  see http://en.wikipedia.org/wiki/FreeDCE
> 
> freedce is an interoperable version of MSRPC that is derived from
> EXACTLY the same source code (DCE 1.0 reference implementation) that
> is present in Windows NT.
> 
> DCE/RPC was co-opted / adopted / borg-ified by microsoft to form
> the basis of NT domains because paul leach, a co-founder of
> apollo computers and originator of NCS which became DCE/RPC,
> ended up working for microsoft back at the time when dave
> cutler was doing the original NT 3.1.
> 
> using FreeDCE is non-essential for small projects of less than about
> 8,000 lines of code.  Samba TNG's MSRPC code comprises OVER ONE HUNDRED 
> THOUSAND lines of code.
> 
> that i carried on hand-crafting MSRPC packets for three years simply
> demonstrates quite how bloody stupid i am.
> 
> that you - the wine team - continue to reinvent an non-interoperable
> version of MSRPC, for binary-level "DCOM" interoperabiltiy ONLY,
> demonstrates quite how just as bloody stupid you are being.  that _can_
> be taken as a compliment, as i genuinely i mean it with the greatest of
> respect.
> 
> 2) regarding a strategy to "minimise" the amount of time needed to
>    "Get Things Working".
> 
> a) utilise samba tng as-it-is, porting it as-is to ReactOS (mingw32)
> b) utilise cli_nt_login_interactive() as-it-is.
> c) complete MSV1_0.DLL.
> 
> this will be SUFFICIENT and would only take a few months.  enhancements
> come later, roughly in this order:
> 
> a) complete a port of FreeDCE to ReactOS, by beating DCEthreads to
>    death and replacing it with Win32 (NT) threads - #ifdef style,
>    autoconf style.
> 
> b) locate NETLOGON.idl, samr.idl and lsarpc.idl from matthew chapman's
>    stuff, or from "todd sabin's" http://razor.bindview.com, or from
>    the samba web site.
> 
> actually... see: http://www.bindview.com/services/razor/utilities/
> 
>    compile up CLIENT-SIDE ONLY, using dceidl, and create a
>    library for use inside MSV1_0.DLL.
> 
>    i know todd developed interoperable versions of these idl files,
>    because he was using them, compiled with MIDL.EXE, to do security
>    tests against NT 4.0.  remember: these IDL files are what microsoft
>    DOESN'T want anyone to have, and there are very good _legitimate_
>    reasons for it: they are "behind-the-scenes" APIs which, if some
>    idiot inside microsoft went and wrote a tool which became publicly
>    used and relied on, they would be stuffed: they have ENOUGH apis
>    which date back 15 years and they don't need any more.  that
>    having been said, i hate the fact that they are forcing people
>    to pay $50k up-front and then $100 per-client for frakkin
>    _information_.  bastards.
> 
> c) replace cli_nt_login_interactive() with a function that utilises
>    the above library that utilises netlogon, samr and lsarpc client-side
>    stuff from b) above.
> 
>    basically, what it boils down to is that the functions inside
>    cli_nt_login_interactive i wrote BY HAND after examining MSRPC
>    traffic.  those functions neeeed too dieeee and they can easily
>    be replaced one-by-one with the "proper" versions from doing
>    "dceidl -client lsarpc.idl".  it takes a couple of seconds and
>    you get a header file lsarpc.h and a lsarpc.o and you're DONE.
> 
>    question.  why did i spend so long doing hand-marshalling of the
>    nt domains code.  perhaps because i would not be able to give
>    microsoft absolute hell for three frakkin years?
> 
> d) consider porting, one-by-one, the Samba TNG services lsarpcd,
>    netlogond and samrd (or better samrtdbd) to FreeDCE runtime
>    infrastructure.
> 
>    this is NOT essential but it is very worthwhile.  every project
>    needs to do at least two throw-aways and with the samba tng
>    code, there has been approximately one and a half throwaways
>    so far... time for a major one, ESPECIALLY in light of the reactos
>    project.
> 
> FINALLY:
> 
> e) consider writing that winregd service but actually picking up REAL
>    registry hive files.  someone did write a "reader" of registry files,
>    i do not recall who it was.
> 
> [note: YESSSS!!! yeeeeeehawww, todd i could KISS you!!]
> 
> 	http://www.bindview.com/Services/RAZOR/Utilities/Unix_Linux/ntreg_readme.cfm
> 
>   todd wrote a registry-hive-reader as a linux filesystem driver!!!!!
> 
>   todd, you are bloody mad.  *smooch*.
> 
> 
> f) consider writing a samr service with the FreeDCE runtime
>    that utilises "registry" functions RegXXXX - _properly_.
> 
> 
> anyway, if you got this far: congratulations and welcome to a brief
> nightmare history of the development behind NT Domains Authentication.
> 
> all of the infrastructure above _already_ exists - in Samba TNG.
> it's just...  it works.  it's a bit creaky at the edges, but it works.
> 
> how about it?
> 
> l.
> 
> p.s. it almost goes without saying, but i will hint at it once again:
> you will not find the samba team's goals and strategic direction to be
> compatible with reactos.  as a team, they lack the insight, vision and
> project management skills to be able to cope with such inter-dependant
> insane gibberish.  this i say with much sadness because they are highly
> respected individuals in the free software community,  and yet they are
> taking the samba project in very different - and ultimately
> time-consuming - directions from what is really really needed.  due
> to pride they cannot admit this in public but they have been known
> to admit it in private.  much much disappointment and sadness and there's
> absolutely nothing i can do about it.
> 
> -- 
> --
> <a href="http://lkcl.net">http://lkcl.net</a>
> --
> _______________________________________________
> Ros-dev mailing list
> Ros-dev at reactos.com
> http://reactos.com:8080/mailman/listinfo/ros-dev
> 


-- 
David Johnson
http://www.davefilms.us