TTY: pty, fix pty counting
authorJiri Slaby <jslaby@suse.cz>
Wed, 10 Aug 2011 12:59:28 +0000 (14:59 +0200)
committerGreg Kroah-Hartman <gregkh@suse.de>
Tue, 23 Aug 2011 17:10:38 +0000 (10:10 -0700)
commit24d406a6bf736f7aebdc8fa0f0ec86e0890c6d24
tree3dfb2053f7785508304fd29d6c1bb6e6a42a7d5c
parentdbb3b1ca5609d1f3848cd387d06cc60aaacf7f98
TTY: pty, fix pty counting

tty_operations->remove is normally called like:
queue_release_one_tty
 ->tty_shutdown
   ->tty_driver_remove_tty
     ->tty_operations->remove

However tty_shutdown() is called from queue_release_one_tty() only if
tty_operations->shutdown is NULL. But for pty, it is not.
pty_unix98_shutdown() is used there as ->shutdown.

So tty_operations->remove of pty (i.e. pty_unix98_remove()) is never
called. This results in invalid pty_count. I.e. what can be seen in
/proc/sys/kernel/pty/nr.

I see this was already reported at:
  https://lkml.org/lkml/2009/11/5/370
But it was not fixed since then.

This patch is kind of a hackish way. The problem lies in ->install. We
allocate there another tty (so-called tty->link). So ->install is
called once, but ->remove twice, for both tty and tty->link. The fix
here is to count both tty and tty->link and divide the count by 2 for
user.

And to have ->remove called, let's make tty_driver_remove_tty() global
and call that from pty_unix98_shutdown() (tty_operations->shutdown).

While at it, let's document that when ->shutdown is defined,
tty_shutdown() is not called.

Signed-off-by: Jiri Slaby <jslaby@suse.cz>
Cc: Alan Cox <alan@linux.intel.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
drivers/tty/pty.c
drivers/tty/tty_io.c
include/linux/tty.h
include/linux/tty_driver.h