Subject: Newbie questions/OSX/compile flags
From: Karen A Swanberg (swanberg@tc.umn.edu)
Date: Wed Oct 03 2001 - 18:09:02 EDT
I just asked over on the developers list if there was more up to date
information on a bunch of the compile flags, and if anyone had an up to
date FAQ yet. Joe Clark sent along the following, with his current
observations on the state of Netatalk. This has lots of good info about
which compile flags to use when, and OSX issues:
I hope to write this, and other things, up for the FAQ, but this way it's
in the archives, which anyone can access at:
ftp ftp://terminator.rs.itd.umich.edu/unix/netatalk/netatalk-admins.mail
(this is a ~6 meg mbox file):
Joe Clark wrote:
I have been using Netatalk now for about four years. I remember
switching between it and the Columbia AppleTalk Package. Eventually, I
settled in with FreeBSD 2.2.6 and Netatalk 1.4b2. My production
envrionment consists of Macintosh-centric advertising agency with about
25 Mac users, and six or so printers.
When I was using 1.4b2, compilation was simple. You just typed
``make,'' and everything built nicely. Eventually, the FreeBSD ports
collection adopted Netatalk, and not soon after, ASun's TCP/DSI patches.
This configuration (Netatalk+asun) worked for a long time.
About six months ago, some of the people in the art department started
complaining that saving Quark Express files caused the server connection
to crash. This happened after they upgraded to Mac OS 9.x. Since I was
unable -- due to lack of Quark -- to reproduce the problem, I searched
to see if the long-awaited next release of ASun's patches were out. To
my suprise, I found Netatalk had been updated to 1.5pre6, and was hosted
on Sourceforge. I downloaded the tar.gz, and tried to get it setup.
The first thing I noticed was that it required GNU configure. Not
knowing what new features were there, I just typed:
./configure
Once that was done, I did a ``make.''. Big suprise, it failed. I then
retired with ``gmake,'' and things started to compile. I ran into some
errors with missing header files, and some improperly defined functions
(improperly defined for FreeBSD anyway). After fixing those errors, I
had a working Netatalk 1.5pre6.
The first thing I noticed was that Quark no longer crashed the server
connection. Now, an occasional ``Error Type -50'' would pop up on the
user's Macintosh when trying to save Quark documents. After scouring
the mailing lists and support forums, I found that this could be worked
around by turning the document preview feature off in Quark. I passed
this information on to the art department, and they quieted down.
However, one day I received a call saying that an entire directory had
been deleted from the server. It wasn't really deleted they said, it
just disappeared. I also noticed some strange errors in the
/var/log/messages file:
WARNING: DID conflict for ... Are these the same file?
Sure enough, the ``...'' was the same name as the missing directory. I
looked everywhere on the filesystem for the directory, but it couldn't
be found. I again went back to the support forums, and found a mention
of using LAST DID to correct this issue. I then rebuilt netatalk with
the configure statement:
./configure --enable-lastdid
This reverted things back to the old 1.4b2 directory ID calculation
algorithm. This also solved the problem of the syslog messages and the
users complaining of file deletions. I also found that by disabling
*BSD's SOFTUPDATES feature on Netatalk volumes, multi-user interaction
seemed to work better. This was back in a late 4.2-BETA, so I'm not
sure if this still holds true in 4.4-RELEASE or not. However, since
this is a production environment, I have left SOFTUPDATES off on my
Netatalk volumes.
When I tried to get Netatalk 1.5pre6 working in Mac OS X, things failed
miserably. No matter what I tried to do, I would get an error. If I
tried to open a document from a Netatalk server in OS X, I'd receive a
``Application for this document not found'' error. If I tried to copy a
file from the Netatalk server to OS X, I'd get an ``Error Type -43.'' I
found information in the Apple discussion forums and in Netatalk's
support forum that said a late 1.4 beta worked well with OS X. I
downloaded this, and found that the LAST DID algorithm had been broken
in certain cases in pre6. I submitted patches to fix it, and voila, OS
X worked with Netatalk. Again, I had to configure with the
--enable-lastdid configure argument.
What about file and directory permissions? Since I didn't use the FORCE
UID/GID code, I decided to use a feature of FreeBSD called SUIDDIR.
>From the LINT kernel config file:
# If you are running a machine just as a fileserver for PC and MAC
# users, using SAMBA or Netatalk, you may consider setting this option
# and keeping all those users' directories on a filesystem that is
# mounted with the suiddir option. This gives new files the same
# ownership as the directory (similar to group). It's a security hole
# if you let these users run programs, so confine it to file-servers
# (but it'll save you lots of headaches in those cases). Root owned
# directories are exempt and X bits are cleared. The suid bit must be
# set on the directory as well; see chmod(1) PC owners can't see/set
# ownerships so they keep getting their toes trodden on. This saves
# you all the support calls as the filesystem it's used on will act as
# they expect: "It's my dir so it must be my file".
FORCE UID/GID code, I decided to use a feature of FreeBSD called
SUIDDIR. From the LINT kernel config file:
# If you are running a machine just as a fileserver for PC and MAC
# users, using SAMBA or Netatalk, you may consider setting this option
# and keeping all those users' directories on a filesystem that is
# mounted with the suiddir option. This gives new files the same
# ownership as the directory (similar to group). It's a security hole
# if you let these users run programs, so confine it to file-servers
# (but it'll save you lots of headaches in those cases). Root owned
# directories are exempt and X bits are cleared. The suid bit must be
# set on the directory as well; see chmod(1) PC owners can't see/set
# ownerships so they keep getting their toes trodden on. This saves
# you all the support calls as the filesystem it's used on will act as
# they expect: "It's my dir so it must be my file".
And the associated mount command:
mount -o suiddir /dev/da2s1e /macvol/artfiles
This was used on my dedicated Netatalk/Samba filesystems. On
filesystems that were also used for interactive shell access, I chmod'd
my Netatalk shares 2770. The reason for this is that I set up a UNIX
group for each department in the ad agency. I had an art group, a media
group, an accounting group, and then, or course, a general staff group.
Each share was only allowed access by the group that needed to access
the share. So, the Artfiles share allowed access only to the art group:
/macvol/artfiles "Art Files" allow:@art
And the others followed in kind. Therefore, the 2770 mask allowed only
owners and people in the associated group access to read and write
files. The leading 2 set the setgid bit so that all child files and
directories would retain the same group permissions. I found this to
work well.
This was used on my dedicated Netatalk/Samba filesystems. On
filesystems that were also used for interactive shell access, I chmod'd
my Netatalk shares 2770. The reason for this is that I set up a UNIX
group for each department in the ad agency. I had an art group, a media
group, an accounting group, and then, or course, a general staff group.
Each share was only allowed access by the group that needed to access
the share. So, the Artfiles share allowed access only to the art group:
/macvol/artfiles "Art Files" allow:@art
And the others followed in kind. Therefore, the 2770 mask allowed only
owners and people in the associated group access to read and write
files. The leading 2 set the setgid bit so that all child files and
directories would retain the same group permissions. I found this to
work well.
After a while running with 1.5pre6, 1.5pre7 was ready to go out. There
wasn't too much I was interested in implementing in this release, but I
was now maintaining the FreeBSD port of Netatalk, and I wanted to keep
my port current. One of the hot new features in pre7 was the new
persistent DID code based on Berkeley db3 by Sleepy Cat Software. With
this code, it would be possible to keep unique DIDs between reboots
afpd. This code was still experimental, however.
The CNID DB code differed from the LAST DID code in the fact that it
used persisten disk-based storage to store filename-to-DID mappings,
dev/inode mappings, and CNID mappings. The algorithm was simple. The
DID starts at the first usable ID (3 in this case), and increments until
a unique ID is found. That is then inserted into the database. If the
ID is later freed up, a newly added directory will take it in its walk
through the database.
There were quite a few users eager to start playing with this new DID
code (called CNID DB). I figured, I'd test it enough to add support for
it to my FreeBSD port. During me testing, I found the code to be mainly
broken. It was development exclusively on Linux using db3 3.1.17.
FreeBSD's ports collection included 3.2.9 as its stable version of db3.
There were quite a few differences between the two APIs. Adding to this
was the fact that Sleepy Cat also had db3 3.3.11 out as well.
In order to make CNID DB not only compile, but work somewhat, I began
analyzing the code. I fixed quite a few bugs, and added some
portability improvements. The code should now work (or at least
compile) on db3 3.1.x, 3.2.x, and 3.3.x. However, I have not been able
to test it thoroughly enough to state it's ready for production. There
still seems to be some locking issues from what I can tell from the
mailing lists.
All of these CNID DB improvements are in the Netatalk CVS, but they are
not in an official release. If you try to use 1.5pre7 without db3
installed (even if you don't plan to use the CNID DB DID code), you will
get compilation issues. This is because the db3 code is compiled no
matter what. Patches were committed to CVS to correct this. Please see
https://sourceforge.net/cvs/?group_id=8642 for more information on
obtaining a recent CVS version.
If you wish to enable the CNID DB DID code, configure Netatalk with:
./configure --enable-cnid-db
For now, I still stick with the LAST DID algorithm in production. I
also compile without PAM support. This requires the --without-pam
configure argument since PAM is built by default. For my FreeBSD port,
I configure using:
./configure --without-pam --pkgconfdir=/usr/local/etc --with-did=last
--with-tcp-wrappers
The --with-tcp-wrappers option tells Netatalk to use TCP Wrappers with
afpd. This allows finer grain control over who can connect to my afpd
DSI/TCP shares. The --with-did=last is just the pre7 incarnation of
--enable-lastdid. The --pkgconfdir option tells Netatalk to use
/usr/local/etc from config files instead of the default
/usr/local/etc/netatalk.
As I continued to listen to users' comments on the netatalk-devel
mailing list, I found that Quark Express continued to be a problem. I
asked my art department again if they still had Quark problems. Some
said no, others said yes. Some Linux users said that another configure
argument, --with-flock-locks, helped them solve their Quark problems
among another things. Enabling this code disabled the new byte locking
feature. With FLOCK locks, the whole file would be locked. With byte
locks, a byte range could be locked without lcoking the whole file. In
any event, this didn't seem to be working correctly on Linux.
I've been trying out --with-flock-locks in my production environment for
a few days now, and so far, so good. They haven't done much Quark work
lately. If they start to experience problems, I'll re-enable byte
locks. Else, I'll stick to the older style 1.4b2 code.
Since 1.5pre7 was released, a few users have contributed patches to
improve one thing or another. For instance, patches were committed to
re-enable codepage translations, fix the custom icon (-I) argument to
afpd, add Service Location Protocol support, and stop seg faults when no
atalkd.conf file exists (to name a few). Others in review are better
advisory locking code when dealing with read-only files as well as
better handling of errors when inaccessible directories are encountered.
Problems that still exist are occasional core dumps when dealing with
certain files (under investigation), CNID DB DID problems (awaiting
feedback from users on latest patches), copy problems from Mac to
Netatalk (trying to reproduce), and better file locking capabilities in
general (Samba to contribute???).
This is just my take on things that I have seen an dealt with. I
apologize if anything is blatantly wrong.
Joe Clarke
marcus@marcuscom.com
- -
Karen Swanberg | Sys Admin | Dept. of Geology and Geophysics
206 Pillsbury Hall | 310 Pillsbury Ave. SE | University of Minnesota
Minneapolis, MN 55455 (612) 624-6541 (612) 625-3819 (f)
* <---- Tribble . <--- Tribble.tgz
This archive was generated by hypermail 2b28 : Sun Oct 14 2001 - 03:04:54 EDT