Pour limiter les utilisateurs ayant le droit de se connecter à un serveur SSH, on peut créer un fichier /etc/sshusers_allowed dans lequel on met les utilisateurs autorisés.

Ensuite, on demande à SSH de vérifier en éditant /etc/pam.d/ssh, et en rajoutant :
auth       required    pam_listfile.so item=user sense=allow file=/etc/sshusers_allowed onerr=fail

Le fichier total peut être :
# PAM configuration for the Secure Shell service

# Read environment variables from /etc/environment and
# /etc/security/pam_env.conf.
auth       required     pam_env.so # [1]
# In Debian 4.0 (etch), locale-related environment variables were moved to
# /etc/default/locale, so read that as well.
auth       required     pam_env.so envfile=/etc/default/locale

# Limitation des utilisateurs ayant le droit de se connecter
auth       required    pam_listfile.so item=user sense=allow file=/etc/sshusers_allowed onerr=fail

# Standard Un*x authentication.
@include common-auth

# Disallow non-root logins when /etc/nologin exists.
account    required     pam_nologin.so

# Uncomment and edit /etc/security/access.conf if you need to set complex
# access limits that are hard to express in sshd_config.
# account  required     pam_access.so

# Standard Un*x authorization.
@include common-account

# Standard Un*x session setup and teardown.
@include common-session

# Print the message of the day upon successful login.
session    optional     pam_motd.so # [1]

# Print the status of the user's mailbox upon successful login.
session    optional     pam_mail.so standard noenv # [1]

# Set up user limits from /etc/security/limits.conf.
session    required     pam_limits.so

# Set up SELinux capabilities (need modified pam)
# session  required     pam_selinux.so multiple

# Standard Un*x password updating.
@include common-password