SAMBA, Cygwin, and appcompat

Here you can discuss ReactOS related topics.

Moderator: Moderator Team

Post Reply
bugdude
Posts: 28
Joined: Wed Nov 27, 2019 11:41 pm

SAMBA, Cygwin, and appcompat

Post by bugdude »

Hello all,

I have been working on creating a static build of Samba 4.11.0 for use in ReactOS through cygwin. I was wondering if there is any interest in this before I spend too much time on it. Getting a workable build environment has been a chore because Cygwin ceased supporting XP/2003 some time ago, and Samba has a longish dependency list making it necessary to build an oldish environment and source build dependencies. I have made good progress already, but the marked difference between doing this and building against the current cygwin also made me wonder about something..

I would like to respectfully propose an argument for special case handling of some NT6 APIs(having read the many threads about this). I think it would be useful to implement a 'select' set of NT6 functions so that projects like cygwin/samba/dokany/winfsp could be used to expand the ReactOS operating system level functionality(where they are license compatible). I think SAMBA is basically the 'poster child' for this idea because is GPL and gives a core OS functionality that is currently missing, and because rebuilding it is a much larger undertaking. Making this kind of exception could allow may projects to be integrated in ROS rather than re-built. This could shorten the list of missing OS features to reach 1.0 quite effectively and reduce the development needed to get there. Getting cygwin running well in ReactOS would not just open the door to a real integration of SAMBA, but also to using many other GPL applications on ROS that lie outside the OS level.

Let me say immediately that I am not a hard core developer (certainly not a kernel dev), but rather a hacker of sorts, so I do not 'pretend' to know the degree of effort involved in doing this. Some of the APIs are present in WINE code and perhaps they could be reused(if they are up to snuff). The higher level APIs could almost certainly be handled through the appcompat work already underway, but I suspect that the lower level items like fsrtl routines or oplock type functions might merit being exposed from the kernel layer in spite of their being NT6 functions..

Opinions ? Feedback ?

bugdude
karlexceed
Posts: 531
Joined: Thu Jan 10, 2013 6:17 pm
Contact:

Re: SAMBA, Cygwin, and appcompat

Post by karlexceed »

It seems counter-productive to try and make Samba work via cygwin. The goal should be to have it work without cygwin and, ideally, without relying on any NT6+ API. Better yet, build it within ROSBE, the ReactOS Build Environment.

The version of "Samba for ReactOS" available via RApps claims to be 1.3, but this isn't likely directly related to Samba release versions, otherwise that would mean it's pre-1993 code. This package says it, "Provides authentication providers for secur32.dll (e.g. NTLM ...) needed for running applications."

The version of Samba TNG available via RApps is 0.5-RC1, which was released in 2009. This package says that it, "Allows you to access Windows shared folders/printers with ReactOS."

So there are useful aspects to both packages, but what we need is all of the functionality in one maintainable package. My understanding is that Samba TNG is/was better structured for integration into ROS. So what we really need, more than a cygwin Frankenstein build of Samba, is a resurrection of the Samba TNG project. Unfortunately, I can't find any snapshots of their codebase from which to start.
bugdude
Posts: 28
Joined: Wed Nov 27, 2019 11:41 pm

Re: SAMBA, Cygwin, and appcompat

Post by bugdude »

karlexceed,

I agree conceptually, but I believe the Samba-TNG project has been silent (or close to it) for a good long while, and seems unlikely to revive any time soon. At the same time Samba has picked up AD functionality, SMB2/3 and encryption capabilities both as client and a server. It can communicate with recent versions of Windows server without re-enabling the SMB1 protocol, and I believe Samba-TNG cannot yet. I have already made working test builds, but I haven't had time to see if more advanced things like AD features, SMB2 and encryption work in these builds.

Building on cygwin means the binaries require a handful of DLLs, and that would make them heavier and slower than a 'native' development. Even so the mapping/marshaling layer in cygwin is fairly thin, so the builds could still prove to be useful. With more time and effort Samba could be 'ported' and built with ROSBE's mingw-gcc rather than cygwins gcc, but that would likely mean porting several dependencies as well.

If you read older threads on Samba-TNG you will see that even that was basically orphaned for not being easily integrated with ReactOS, so the ROSAPPs package you refer to is really just a standalone command line client binary. I just thought doing this might spark some movement for networking under ROS. Additionally I thought this could allow the ROS development team to concentrate on the core OS/Kernel for which no such 'shortcuts' are likely to ever exist.

This is a bit like the first steps of merging WINE code, a tentative first step.

bugdude
User avatar
EmuandCo
Developer
Posts: 4723
Joined: Sun Nov 28, 2004 7:52 pm
Location: Germany, Bavaria, Steinfeld
Contact:

Re: SAMBA, Cygwin, and appcompat

Post by EmuandCo »

There already were some efforts porting Samba. Check rapps and you will find a minimal port to get Office 2010 do work rather ok. Maybe that can help your on your journey. It was only made to provide some encryption algorithms needed, but it was a start
ReactOS is still in alpha stage, meaning it is not feature-complete and is recommended only for evaluation and testing purposes.

If my post/reply offends or insults you, be sure that you know what sarcasm is...
karlexceed
Posts: 531
Joined: Thu Jan 10, 2013 6:17 pm
Contact:

Re: SAMBA, Cygwin, and appcompat

Post by karlexceed »

By all means, do what you can to get this working - I don't intend to try and dissuade you at all. The Samba TNG release from 2009 that I mentioned was about the same time that the project died.

What I meant by suggesting resurrecting the Samba TNG project was simply that I had understood it's structure to be more modular and that it was built to run natively in ReactOS/Win32 - both things that would be advantageous. I was wondering about the possibility of finding that code and updating it with the changes made in Samba proper, and whether or not it would be able to save some work.

And lastly, in terms of the functionality - remember that we want to be compatible with Server 2003 SP3 / XP in the short term, while being able to move to a more modern target in the future. This means that we would probably want to have SMB1 enabled by default for now among other things.
bugdude
Posts: 28
Joined: Wed Nov 27, 2019 11:41 pm

Re: SAMBA, Cygwin, and appcompat

Post by bugdude »

Emuanco,

Thanks, I will certainly look at both of them. Since RosBE uses GNU toolchain I do hope to shift things into that once I have a proof of concept going. This is just a baby step seeing if perhaps some services could be built in pilot form at the application layer so they could later be fused into ROS once they were adapted for a better fit. Samba does not use the API calls and layers windows uses, so I doubt a simple merge would ever be possible, but Samba does have most or all of the functionality needed to build the networking layers missing in ROS.

karlexceed,

I started by looking for samba-tng end had enough trouble finding code that I gave up on it. I believe some of the concepts proposed in TNG made their way back into Samba over time (it is now using a dce/rpc/idl compiler rather than hand-coding routines where possible). Since the Samba code base is pretty large I believe that process is an ongoing activity. I suspect that to some degree the 'modular' structure proposed in TNG was started by defining new targets and linking selectively using the existing code, a process which could be repeated with current code. That was a level of analysis I had not considered yet. That would be limited by the existing dependencies in the code, but 'perhaps' it would permit segregating the components into the 'layers' present in windows networking. That would be a first step to really integrating because it could allow some 'overlap' to be eliminated and perhaps tie the code in more tightly. Bridging from TNG as you describe is probably also possible, but if the code base is old enough it may not prove to be a better choice.

The SMB levels are configurable to a degree so SMB1 can probably be re-enabled, but that is definitely a place where a balance needs to be struck between the development target and security/technology changes. Should ROS replicate a 'well known bug' just because it 'existed' (not an errant API behavior, but rather something with potential for data damage, remote vulnerability or DOS attack). I suppose if key software depended on the behavior, re-creating it might be justified, but it is definitely a good place to think twice. Is the ultimate goal being functional and API compatible, or being indistinguishable including replicating major vulnerabilities that were ultimately eliminated (WannaCry uses SMB1 weaknesses) ?

bugdude
petr-akhlamov
Posts: 60
Joined: Wed Apr 10, 2013 3:23 pm
Location: Russia, Moscow

Re: SAMBA, Cygwin, and appcompat

Post by petr-akhlamov »

EmuandCo wrote: Tue Dec 03, 2019 2:25 am There already were some efforts porting Samba. Check rapps and you will find a minimal port to get Office 2010 do work rather ok. Maybe that can help your on your journey. It was only made to provide some encryption algorithms needed, but it was a start
This port quite good perform yourself some functions.

https://reactos.org/wiki/User:Petr-akhlamov/Samba

http://youtu.be/pGpXKA3EqJM
User avatar
binarymaster
Posts: 481
Joined: Sun Nov 16, 2014 7:05 pm
Location: Russia, Moscow
Contact:

Re: SAMBA, Cygwin, and appcompat

Post by binarymaster »

bugdude wrote: Fri Nov 29, 2019 11:32 pm I have been working on creating a static build of Samba 4.11.0 for use in ReactOS through cygwin. I was wondering if there is any interest in this before I spend too much time on it.
Hello bugdude!

Please check out ReactOS Chat development channel, some work discussed here, which could collide with your work, or is highly related: https://chat.reactos.org/reactos/pl/pim ... zj9tbd1gbo
karlexceed
Posts: 531
Joined: Thu Jan 10, 2013 6:17 pm
Contact:

Re: SAMBA, Cygwin, and appcompat

Post by karlexceed »

shunesburg
Posts: 215
Joined: Wed Feb 21, 2018 3:46 pm
Location: Somewhere in France

Re: SAMBA, Cygwin, and appcompat

Post by shunesburg »

A native version of Samba 4.11 would be better. I wonder why they didn't make one for Windows?
karlexceed
Posts: 531
Joined: Thu Jan 10, 2013 6:17 pm
Contact:

Re: SAMBA, Cygwin, and appcompat

Post by karlexceed »

Presumably because the SMB support comes built in...
erkinalp
Posts: 861
Joined: Sat Dec 20, 2008 5:55 pm
Location: Izmir, TR

Re: SAMBA, Cygwin, and appcompat

Post by erkinalp »

bugdude wrote:Is the ultimate goal being functional and API compatible, or being indistinguishable including replicating major vulnerabilities that were ultimately eliminated
It is something between the two, according to previous discussions. The problem is we just do not know how much we can modify or extend Windows API without breaking the compatibility too much.
-uses Ubuntu+GNOME 3 GNU/Linux
-likes Free (as in freedom) and Open Source Detergents
-favors open source of Windows 10 under GPL2
shunesburg
Posts: 215
Joined: Wed Feb 21, 2018 3:46 pm
Location: Somewhere in France

Re: SAMBA, Cygwin, and appcompat

Post by shunesburg »

karlexceed wrote: Mon Dec 09, 2019 9:03 pm Presumably because the SMB support comes built in...
Yes and no, the client and server are present, but the Active Directory features of Samba are only built in on Windows Server only, not for Windows Desktop.
bugdude
Posts: 28
Joined: Wed Nov 27, 2019 11:41 pm

Re: SAMBA, Cygwin, and appcompat

Post by bugdude »

Hello again,

I have been working at this slowly because the conversation here put on a slightly different track. Originally I was just trying to get a static build that left the posix layer components of cygwin and most of SAMBA's dependencies as shared libraries. I had that build working when I started this discussion, but the comments made here caused me to shift my plans and trying to get as many of the dependencies built statically as well. I have made progress with that larger goal, which would create binaries requiring just one or two DLLs (Cygwin1 and Python mainly).

The idea turned out to be more work than I first thought because several of the elements that SAMBA depends on are shared libraries with further dependencies of their own making it necessary to crawl slowly down the dependency chain replacing several libraries at multiple layers each time. It also got more ornate because cygwin and linux will prefer and use a shared library if both shared and static ones exist.. SAMBA is also not really designed with fully static linking in mind. The result is that every time I replace a library I have to make some changes in the Samba waf based build system, as well as removing the shared library objects. I also foudn that removing the shared objects occasionally breaks components of the build toolchain SAMBA relies on requiring them to be rebuilt statically as well. Once I have an almost static binary I will look at finding a way to statically link the cygwin1 itself, which is now allowed under its license terms. If I can get that far I hope to be able to follow up by combining the results with something like DOKANY and SMBNET-FS in order to make a windows type IFS driver/redirector and make an SMB network client for ROS.

I did consider the idea of just building a client redirector from scratch within ROS, but to be honest I didn't want to have to learn the entire SAMBA codebase as well as the windows IFS layer just to get started. Even though this method is a 'thing-on-a-thing' aimed at geting an SMB client, it allows me to work single handed in small chunks in my spare time.. Developing a full blown client layer for ROS from zero just looked like a more difficult undertaking to me.

What I have learned along the way is that the 4.x codebase for SAMBA does make heavier use of generic RPC call mapping mechanisms and generic routines than the 3.x code did, so now think there's a real possibility of fusing the 4.x components to the existing ROS networking pieces and create an integrated client for ROS. I'm not sure 'I' would be able to do that, but I am starting to think that it is a viable idea.

The downside of this approach is that I am swapping shared components that have some degree of testing for newly built versions, so there is a higher potential for injecting problems. Since I have been focused on hacking this together I have not been able to do much testing yet, but the build using shared objects worked in a workgroup networking environment against Windows 10 shares without enabling SMB1 when I tried it. I never mentioned it earlier, but I am trying to keep the entire SAMBA code tree building, so I am building AD file servers that should run on ROS as I work.

Someone asked why the SAMBA project doesn't create a Windows version, and karlexceed mentioned the primary reason, because the functionality is built into windows, but even so in the SAMBA lists there are comments from people who were interested in grafting SAMBA onto Windows XP in the interest of repairing some of the issues in its SMB stack, but it never went anywhere because they found there was no way to get Windows to surrender its exclusive lock on the default ports required to do that, so there is also a technical reason nobody tries to build for Windows as well.

Bugdude
Post Reply

Who is online

Users browsing this forum: Google [Bot] and 43 guests