can't login when LDAP server is down

[bmarshallbri]

Hi All,

I'm having a problem with LDAP and nscd caching. It is my understanding (correct me if I'm wrong) that one of the features of NSCD is to cache ldap passwd info in the event the ldap server is unavailable. Since I can't seem to get it working I am assuming that I am doing something wrong.

I have a test LDAP server and a test client server that I can login as a non-local user via LDAP but when I shut the LDAP server down the client server fails authentication. Local users can still login on the client server. But what I am trying to accomplish is ultimately servers and laptops that can operate in the absence of a network or in the event of an LDAP server failure.

Some of my config info is below. Please let me know if you want to see anything specific on my configs. Any help would be greatly appreciated.

Thanks

Brian

authconfig --test

[code]
caching is enabled
nss_files is always enabled
nss_compat is disabled
nss_db is disabled
nss_hesiod is disabled
hesiod LHS = ""
hesiod RHS = ""
nss_ldap is enabled
LDAP+TLS is disabled
LDAP server = "ldap://10.179.99.154/"
LDAP base DN = "dc=xxxxxxxxxxx,dc=com"
nss_nis is disabled
NIS server = ""
NIS domain = ""
nss_nisplus is disabled
nss_winbind is disabled
SMB workgroup = "WORKGROUP"
SMB servers = ""
SMB security = "user"
SMB realm = ""
Winbind template shell = "/bin/false"
SMB idmap uid = "16777216-33554431"
SMB idmap gid = "16777216-33554431"
nss_wins is disabled
pam_unix is always enabled
shadow passwords are enabled
password hashing algorithm is descrypt
pam_krb5 is disabled
krb5 realm = "EXAMPLE.COM"
krb5 realm via dns is disabled
krb5 kdc = "kerberos.example.com:88"
krb5 kdc via dns is disabled
krb5 admin server = "kerberos.example.com:749"
pam_ldap is enabled

LDAP+TLS is disabled
LDAP server = "ldap://10.179.99.154/"
LDAP base DN = "dc=xxxxxxxxxxxx,dc=com"
pam_pkcs11 is disabled

use only smartcard for login is disabled
smartcard module = "None"
smartcard removal action = ""
pam_smb_auth is disabled
SMB workgroup = "WORKGROUP"
SMB servers = ""
pam_winbind is disabled
SMB workgroup = "WORKGROUP"
SMB servers = ""
SMB security = "user"
SMB realm = ""
pam_cracklib is enabled (try_first_pass retry=3)
pam_passwdqc is disabled ()
pam_access is disabled ()
pam_mkhomedir is disabled ()
Always authorize local users is enabled ()
Authenticate system accounts against network services is disabled
[/code]

[Moderator edited to insert [i]code[/i] tags to preserve the format and aid readability.]

[scottro]

HEY! That bug is only 5 or 6 years old, why would you expect it to be fixed. :-)

See my page http://home.roadrunner.com/~computertaijutsu/ldap.html

You have to make a slight change somewhere--do a search for the word "soft" on the page. That should fix the problem.

[bmarshallbri]

Ha :-) That's funny. I actually have been doing my testing based on some of your article. Good article. Thanks for taking the time to put that together.

I do already have 'bind_policy soft' in /etc/ldap.conf. It doesn't seem to make a difference. hard/soft I still can't login when the LDAP server is offline.

Are there any other things you can think of that I need to check in my configs?

[scottro]

Hrrm, that's usually fixed it. Of course, it's CentOS, based on RH, so I always expect it to be bad.

I'm afraid I'm stuck.

Did you definitely put files ldap in /etc/nsswitch.conf? (If I've overlooked it, I apologize, but I don't see it mentioned in your earlier post.)

[AlanBartlett]

[quote]
HEY! That bug is only 5 or 6 years old, why would you expect it to be fixed. :-)
[/quote]
[b]Scott[/b] -- Do you have an [url=https://bugzilla.redhat.com/frontpage.cgi]upstream bug tracker reference[/url] logging this issue for [b]RHEL 5[/b]? If one does not exist then, as far as the maintainers are concerned, the issue [i][b]does not exist[/b][/i]. Grumbling here about [url=http://wiki.centos.org/FAQ/General?highlight=%28TUV%29#head-d29a2b7e61ffc544973098f9dd49fe4663efba50]TUV[/url] won't get things fixed [i]upstream[/i]. :roll:

[scottro]

Hrrm, thought I referenced it in the article, but I see I didn't. Ah well, I will have to look later today if I have a chance.

[AlanBartlett]

Thanks! ( And, in friendship, here's the :-P I omitted to affix after my :roll: , above. :lol: )

[bmarshallbri]

Hi scottro,

Yeah I just double checked to make sure. Below is my /etc/nsswitch.conf. I'm stuck on this one. At this point I have not configured TLS. That can't have anything to do with it does it?

[code]
passwd: files ldap
shadow: files ldap
group: files ldap
hosts: files dns
bootparams: nisplus [NOTFOUND=return] files
ethers: files
netmasks: files
networks: files
protocols: files
rpc: files
services: files
netgroup: files ldap
publickey: nisplus
automount: files ldap
aliases: files nisplus
[/code]


Also, do you know where NSCD caches the LDAP data? I'd like to see if it's even writing a cache file anywhere.

I just checked the article again. It does reference a RH bug reported but does not link to it. Can you provide me with that bug information so I can bump it?

[bmarshallbri]

Any ideas anyone?

I have worked in plenty of corporate environments with Mac's and Windows machines that are capable of caching LDAP info for logins in the absence of a network. I have to believe that since LDAP came out of the unix world that unix/linux had to have solved this problem long before Apple switched to Mach BSD and Windows subjected us all to NT 4.0.

I'm reaching out to you all you *nix heads. Can anyone show me the errors of my ways? :hammer:

For good measure; here's some detailed info about my configuration and some system log snippets from tests.

/etc/nsswitch.conf
[code]
passwd: files ldap
shadow: files ldap
group: files ldap
hosts: files dns
bootparams: nisplus [NOTFOUND=return] files
ethers: files
netmasks: files
networks: files
protocols: files
rpc: files
services: files
netgroup: files ldap
publickey: nisplus
automount: files ldap
aliases: files nisplus
[/code]

/etc/nscd.conf looks like this
[code]
logfile /var/log/nscd.log
debug-level 5
server-user nscd
paranoia no

enable-cache passwd yes
positive-time-to-live passwd 3600
negative-time-to-live passwd 20
suggested-size passwd 211
check-files passwd yes
persistent passwd yes
shared passwd yes
max-db-size passwd 33554432
auto-propagate passwd yes

enable-cache group yes
positive-time-to-live group 3600
negative-time-to-live group 60
suggested-size group 211
check-files group yes
persistent group yes
shared group yes
max-db-size group 33554432
auto-propagate group yes

enable-cache hosts yes
positive-time-to-live hosts 3600
negative-time-to-live hosts 20
suggested-size hosts 211
check-files hosts yes
persistent hosts yes
shared hosts yes
max-db-size hosts 33554432
[/code]

And my /etc/ldap.conf is this
[code]
ssl no
tls_cacertdir /etc/openldap/cacerts
pam_password crypt
uri ldap://xxx.xxx.xxx.xxx/
base dc=xxxxxxxxxx,dc=com
bind_policy soft
[/code]

here's /etc/pam.d/system-auth
[code]
auth required pam_env.so
auth sufficient pam_unix.so nullok try_first_pass
auth requisite pam_succeed_if.so uid >= 500 quiet
auth sufficient pam_ldap.so use_first_pass
auth required pam_deny.so

account required pam_unix.so broken_shadow
account sufficient pam_localuser.so
account sufficient pam_succeed_if.so uid < 500 quiet
account [default=bad success=ok user_unknown=ignore] pam_ldap.so
account required pam_permit.so

password requisite pam_cracklib.so try_first_pass retry=3
password sufficient pam_unix.so shadow nullok try_first_pass use_authtok
password sufficient pam_ldap.so use_authtok
password required pam_deny.so

session optional pam_keyinit.so revoke
session required pam_limits.so
session [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid
session required pam_unix.so
session optional pam_ldap.so
[/code]

here's what nscd is logging when I perform an ssh login as an LDAP user with the LDAP server up (as in I can connect successfully).

[b]Note that my username is "tester" and I am testing login against sshd "ssh -p MY_SSH_PORT tester@mytestserver"[/b]

[code]
2010: handle_request: request received (Version = 2) from PID 2022
2010: GETFDPW
2010: provide access to FD 6, for passwd
2010: handle_request: request received (Version = 2) from PID 2022
2010: GETPWBYNAME (sshd)
2010: Haven't found "sshd" in password cache!
2010: add new entry "sshd" of type GETPWBYNAME for passwd to cache (first)
2010: add new entry "74" of type GETPWBYUID for passwd to cache
2010: handle_request: request received (Version = 2) from PID 2022
2010: GETFDHST
2010: provide access to FD 10, for hosts
2010: handle_request: request received (Version = 2) from PID 2022
2010: GETHOSTBYADDR (67.50.153.210)
2010: Haven't found "67.50.153.210" in hosts cache!
2010: add new entry "67.50.153.210" of type GETHOSTBYADDR for hosts to cache (first)
2010: handle_request: request received (Version = 2) from PID 2022
2010: GETFDGR
2010: provide access to FD 8, for group
2010: handle_request: request received (Version = 2) from PID 2022
2010: INITGROUPS (tester)
2010: Haven't found "tester" in group cache!
2010: add new entry "tester" of type INITGROUPS for group to cache (first)
2010: handle_request: request received (Version = 2) from PID 2022
2010: GETPWBYUID (0)
2010: Haven't found "0" in password cache!
2010: add new entry "0" of type GETPWBYUID for passwd to cache (first)
2010: add new entry "root" of type GETPWBYNAME for passwd to cache
2010: handle_request: request received (Version = 2) from PID 2022
2010: GETGRBYNAME (tty)
2010: Haven't found "tty" in group cache!
2010: add new entry "tty" of type GETGRBYNAME for group to cache (first)
2010: add new entry "5" of type GETGRBYGID for group to cache
2010: handle_request: request received (Version = 2) from PID 2025
2010: GETFDPW
2010: provide access to FD 6, for passwd
2010: handle_request: request received (Version = 2) from PID 2027
2010: GETFDPW
2010: provide access to FD 6, for passwd
2010: handle_request: request received (Version = 2) from PID 2044
2010: GETFDGR
2010: provide access to FD 8, for group
2010: handle_request: request received (Version = 2) from PID 2044
2010: GETGRBYGID (501)
2010: Haven't found "501" in group cache!
2010: add new entry "501" of type GETGRBYGID for group to cache (first)
2010: add new entry "tester" of type GETGRBYNAME for group to cache
2010: handle_request: request received (Version = 2) from PID 2046
2010: GETFDPW
2010: provide access to FD 6, for passwd=
[/code]


And here is what nscd logs when I shut down the LDAP server and try to login to the client machine as "tester" again

[b]Note that nscd doesn't seem to even attempt to look for passwd or group info. It only looks for a host entry for the IP I am connecting from (67.50.153.210)[/b]

[code]
2010: handle_request: request received (Version = 2) from PID 2048
2010: GETFDPW
2010: provide access to FD 6, for passwd
2010: handle_request: request received (Version = 2) from PID 2048
2010: GETFDHST
2010: provide access to FD 10, for hosts
2010: handle_request: request received (Version = 2) from PID 2048
2010: GETHOSTBYADDR (67.50.153.210)
2010: Haven't found "67.50.153.210" in hosts cache!
2010: add new entry "67.50.153.210" of type GETHOSTBYADDR for hosts to cache (first)
2010: handle_request: request received (Version = 2) from PID 2048
2010: GETFDGR
2010: provide access to FD 8, for group
2010: pruning hosts cache; time 1279125445
2010: considering GETAI entry "argentine.summitcove.com", timeout 1279126687
2010: considering GETHOSTBYNAME entry "argentine.summitcove.com", timeout 1279128404
2010: considering GETHOSTBYADDR entry "67.50.153.210", timeout 1279125445
2010: considering GETAI entry "centosz3.centos.org", timeout 1279128237
2010: considering GETAI entry "centosf3.centos.org", timeout 1279126198
2010: pruning hosts cache; time 1279125461
2010: considering GETAI entry "argentine.summitcove.com", timeout 1279126687
2010: considering GETHOSTBYNAME entry "argentine.summitcove.com", timeout 1279128404
2010: considering GETHOSTBYADDR entry "67.50.153.210", timeout 1279125445
2010: considering GETAI entry "centosz3.centos.org", timeout 1279128237
2010: considering GETAI entry "centosf3.centos.org", timeout 1279126198
2010: remove GETHOSTBYADDR entry "67.50.153.210"
[/code]


I also had a fleeting random thought that maybe I had to tell ssh to only use challengeresponse to force it through PAM. That didn't work either. I played around with enabling/disabling combination's of GSSAPI, Password and ChallengeResponse to no avail.

Signed,

A sad keyboard monkey

[bmarshallbri]

Still no love.

I did test "getent passwd tester" and "getent shadow tester" with the LDAP server up and down. "tester" shows up in passwd regardless of the LDAP servers availability (I'm assuming this means cached). "tester" only shows up in shadow when the LDAP server is available (I'm assuming not cached or not referenced).

Any ideas?