GitHub/exynos8895/android_kernel_samsung_universal8895.git
14 years agoceph: use ceph_pagelist for mds reconnect message; change encoding (protocol change)
Sage Weil [Wed, 23 Dec 2009 20:21:51 +0000 (12:21 -0800)]
ceph: use ceph_pagelist for mds reconnect message; change encoding (protocol change)

Use the ceph_pagelist to encode the MDS reconnect message.  We change the
message encoding (protocol change!) at the same time to make our life
easier (we don't know how many snaprealms we have when we start encoding).

An empty message implies the session is closed/does not exist.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: support ceph_pagelist for message payload
Sage Weil [Wed, 23 Dec 2009 20:12:31 +0000 (12:12 -0800)]
ceph: support ceph_pagelist for message payload

The ceph_pagelist is a simple list of whole pages, strung together via
their lru list_head.  It facilitates encoding to a "buffer" of unknown
size.  Allow its use in place of the ceph_msg page vector.

This will be used to fix the huge buffer preallocation woes of MDS
reconnection.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: add feature bits to connection handshake (protocol change)
Sage Weil [Wed, 23 Dec 2009 17:30:21 +0000 (09:30 -0800)]
ceph: add feature bits to connection handshake (protocol change)

Define supported and required feature set.  Fail connection if the server
requires features we do not support (TAG_FEATURES), or if the server does
not support features we require.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: include transaction id in ceph_msg_header (protocol change)
Sage Weil [Tue, 22 Dec 2009 19:24:33 +0000 (11:24 -0800)]
ceph: include transaction id in ceph_msg_header (protocol change)

Many (most?) message types include a transaction id.  By including it in
the fixed size header, we always have it available even when we are unable
to allocate memory for the (larger, variable sized) message body.  This
will allow us to error out the appropriate request instead of (silently)
dropping the reply.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: more informative msgpool errors
Sage Weil [Tue, 22 Dec 2009 18:45:18 +0000 (10:45 -0800)]
ceph: more informative msgpool errors

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: control access to page vector for incoming data
Sage Weil [Tue, 22 Dec 2009 18:45:45 +0000 (10:45 -0800)]
ceph: control access to page vector for incoming data

When we issue an OSD read, we specify a vector of pages that the data is to
be read into.  The request may be sent multiple times, to multiple OSDs, if
the osdmap changes, which means we can get more than one reply.

Only read data into the page vector if the reply is coming from the
OSD we last sent the request to.  Keep track of which connection is using
the vector by taking a reference.  If another connection was already
using the vector before and a new reply comes in on the right connection,
revoke the pages from the other connection.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: use connection mutex to protect read and write stages
Sage Weil [Tue, 22 Dec 2009 18:43:42 +0000 (10:43 -0800)]
ceph: use connection mutex to protect read and write stages

Use a single mutex (previously out_mutex) to protect both read and write
activity from concurrent ceph_con_* calls.  Drop the mutex when doing
callbacks to avoid nested locking (the callback may need to call something
like ceph_con_close).

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: unregister canceled/timed out osd requests
Sage Weil [Tue, 22 Dec 2009 18:29:39 +0000 (10:29 -0800)]
ceph: unregister canceled/timed out osd requests

Canceled or timed out osd requests were getting left in the request list
and never deallocated (until umount).  Unregister if they are canceled
(control-c) or time out.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: only unregister registered bdi
Sage Weil [Tue, 22 Dec 2009 05:04:26 +0000 (21:04 -0800)]
ceph: only unregister registered bdi

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: do not touch_caps while iterating over caps list
Sage Weil [Tue, 22 Dec 2009 04:40:34 +0000 (20:40 -0800)]
ceph: do not touch_caps while iterating over caps list

Avoid confusing iterate_session_caps(), flag the session while we are
iterating so that __touch_cap does not rearrange items on the list.

All other modifiers of session->s_caps do so under the protection of
s_mutex.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: fix incremental osdmap pg_temp decoding bug
Sage Weil [Tue, 22 Dec 2009 00:02:37 +0000 (16:02 -0800)]
ceph: fix incremental osdmap pg_temp decoding bug

An incremental pg_temp wasn't being decoded properly (wrong bound on
for loop).

Also remove unused local variable, while we're at it.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: fix error paths for corrupt osdmap messages
Sage Weil [Mon, 21 Dec 2009 22:49:37 +0000 (14:49 -0800)]
ceph: fix error paths for corrupt osdmap messages

Both osdmap_decode() and osdmap_apply_incremental() should never return
NULL.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: do not drop lease during revalidate
Sage Weil [Mon, 21 Dec 2009 19:48:44 +0000 (11:48 -0800)]
ceph: do not drop lease during revalidate

We need to hold session s_mutex for __ceph_mdsc_drop_dentry_lease(), which
we don't, so skip it.  It was purely an optimization.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: ensure rename target dentry fails revalidation
Sage Weil [Mon, 21 Dec 2009 19:42:18 +0000 (11:42 -0800)]
ceph: ensure rename target dentry fails revalidation

This works around a bug in vfs_rename_dir() that rehashes the target
dentry.  Ensure such dentries always fail revalidation by timing out the
dentry lease and kicking it out of the current directory lease gen.

This can be reverted when the vfs bug is fixed.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: writeback congestion control
Yehuda Sadeh [Fri, 18 Dec 2009 21:51:57 +0000 (13:51 -0800)]
ceph: writeback congestion control

Set bdi congestion bit when amount of write data in flight exceeds adjustable
threshold.

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: writepage grabs and releases inode
Yehuda Sadeh [Wed, 16 Dec 2009 22:51:06 +0000 (14:51 -0800)]
ceph: writepage grabs and releases inode

Fixes a deadlock that is triggered due to kswapd,
while the page was locked and the iput couldn't tear
down the address space.

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
14 years agoceph: remove unaccessible code
Yehuda Sadeh [Wed, 16 Dec 2009 22:22:17 +0000 (14:22 -0800)]
ceph: remove unaccessible code

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
14 years agoceph: include link to bdi in debugfs
Sage Weil [Tue, 15 Dec 2009 22:44:32 +0000 (14:44 -0800)]
ceph: include link to bdi in debugfs

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: make mds ops interruptible
Sage Weil [Tue, 15 Dec 2009 18:27:48 +0000 (10:27 -0800)]
ceph: make mds ops interruptible

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: plug leak of incoming message during connection fault/close
Sage Weil [Fri, 11 Dec 2009 17:48:05 +0000 (09:48 -0800)]
ceph: plug leak of incoming message during connection fault/close

If we explicitly close a connection, or there is a socket error, we need
to drop any partially received message.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: hex dump corrupt server data to KERN_DEBUG
Sage Weil [Mon, 14 Dec 2009 23:13:47 +0000 (15:13 -0800)]
ceph: hex dump corrupt server data to KERN_DEBUG

Also, print fsid using standard format, NOT hex dump.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: fix msgpool reservation leak
Yehuda Sadeh [Tue, 15 Dec 2009 17:50:36 +0000 (09:50 -0800)]
ceph: fix msgpool reservation leak

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
14 years agoceph: don't save sent messages on lossy connections
Sage Weil [Mon, 14 Dec 2009 22:58:11 +0000 (14:58 -0800)]
ceph: don't save sent messages on lossy connections

For lossy connections we drop all state on socket errors, so there is no
reason to keep sent ceph_msg's around.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: detect lossy state of connection
Sage Weil [Mon, 14 Dec 2009 22:56:56 +0000 (14:56 -0800)]
ceph: detect lossy state of connection

The server indicates whether a connection is lossy; set our LOSSYTX bit
appropriately.  Do not set lossy bit on outgoing connections.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: plug msg leak in con_fault
Sage Weil [Mon, 14 Dec 2009 22:30:34 +0000 (14:30 -0800)]
ceph: plug msg leak in con_fault

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoceph: carry explicit msg reference for currently sending message
Sage Weil [Mon, 14 Dec 2009 22:04:30 +0000 (14:04 -0800)]
ceph: carry explicit msg reference for currently sending message

Carry a ceph_msg reference for connection->out_msg.  This will allow us to
make out_sent optional.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: fix leak of monc mutex
Sage Weil [Fri, 11 Dec 2009 16:55:23 +0000 (08:55 -0800)]
ceph: fix leak of monc mutex

Fix leak of monc mutex on ENOMEM or bad fsid when receiving new mon map.
Audited all other users.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: do not feed bad device ids to crush
Sage Weil [Wed, 9 Dec 2009 20:34:01 +0000 (12:34 -0800)]
ceph: do not feed bad device ids to crush

Do not feed bad (large) device ids to CRUSH.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: use kref for ceph_msg
Sage Weil [Mon, 7 Dec 2009 23:55:05 +0000 (15:55 -0800)]
ceph: use kref for ceph_msg

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: use kref for ceph_osd_request
Sage Weil [Mon, 7 Dec 2009 21:37:03 +0000 (13:37 -0800)]
ceph: use kref for ceph_osd_request

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: use kref for struct ceph_mds_request
Sage Weil [Mon, 7 Dec 2009 20:31:09 +0000 (12:31 -0800)]
ceph: use kref for struct ceph_mds_request

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: simplify ceph_buffer interface
Sage Weil [Mon, 7 Dec 2009 20:17:17 +0000 (12:17 -0800)]
ceph: simplify ceph_buffer interface

We never allocate the ceph_buffer and buffer separtely, so use a single
constructor.

Disallow put on NULL buffer; make the caller check.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: use kref for ceph_buffer
Sage Weil [Sat, 5 Dec 2009 18:13:33 +0000 (10:13 -0800)]
ceph: use kref for ceph_buffer

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: mark v0.18 release
Sage Weil [Fri, 4 Dec 2009 18:27:17 +0000 (10:27 -0800)]
ceph: mark v0.18 release

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: update MAINTAINERS entry with correct git URL
Sage Weil [Thu, 3 Dec 2009 23:04:08 +0000 (15:04 -0800)]
ceph: update MAINTAINERS entry with correct git URL

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: allow preferred osd to be get/set via layout ioctl
Sage Weil [Wed, 2 Dec 2009 22:42:39 +0000 (14:42 -0800)]
ceph: allow preferred osd to be get/set via layout ioctl

There is certainly no reason not to report this.

The only real downside to allowing the user to set it is that you don't
get default values by zeroing the layout struct (the default is -1).

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: hide /.ceph from readdir results
Sage Weil [Wed, 2 Dec 2009 19:54:25 +0000 (11:54 -0800)]
ceph: hide /.ceph from readdir results

We need to skip /.ceph in (cached) readdir results, and exclude "/.ceph"
from the cached ENOENT lookup check.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: whitespace cleanup
Sage Weil [Tue, 1 Dec 2009 22:12:07 +0000 (14:12 -0800)]
ceph: whitespace cleanup

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: plug leak of request_mutex
Sage Weil [Tue, 1 Dec 2009 20:23:54 +0000 (12:23 -0800)]
ceph: plug leak of request_mutex

Fix leak of osd client request_mutex on receiving dup ack.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: remove useless IS_ERR checks
Sage Weil [Sat, 21 Nov 2009 21:08:14 +0000 (13:08 -0800)]
ceph: remove useless IS_ERR checks

ceph_lookup_snap_realm either returns a valid pointer or NULL; there is no
need to check IS_ERR(result).

Reported-by: Julia Lawall <julia@diku.dk>
Signed-off-by: Sage Weil <sage@newdream.net>
15 years agofs/ceph: Move a dereference below a NULL test
Julia Lawall [Sat, 21 Nov 2009 15:53:16 +0000 (16:53 +0100)]
fs/ceph: Move a dereference below a NULL test

If the NULL test is necessary, then the dereference should be moved below
the NULL test.

The semantic patch that makes this change is as follows:
(http://coccinelle.lip6.fr/).

// <smpl>
@@
type T;
expression E;
identifier i,fld;
statement S;
@@

- T i = E->fld;
+ T i;
  ... when != E
      when != i
  if (E == NULL) S
+ i = E->fld;
// </smpl>

Signed-off-by: Julia Lawall <julia@diku.dk>
Signed-off-by: Sage Weil <sage@newdream.net>
15 years agofs/ceph: introduce missing kfree
Julia Lawall [Sat, 21 Nov 2009 11:53:08 +0000 (12:53 +0100)]
fs/ceph: introduce missing kfree

Error handling code following a kmalloc should free the allocated data.

The semantic match that finds this problem is as follows:
(http://www.emn.fr/x-info/coccinelle/)

// <smpl>
@r exists@
local idexpression x;
statement S;
expression E;
identifier f,f1,l;
position p1,p2;
expression *ptr != NULL;
@@

x@p1 = \(kmalloc\|kzalloc\|kcalloc\)(...);
...
if (x == NULL) S
<... when != x
     when != if (...) { <+...x...+> }
(
x->f1 = E
|
 (x->f1 == NULL || ...)
|
 f(...,x->f1,...)
)
...>
(
 return \(0\|<+...x...+>\|ptr\);
|
 return@p2 ...;
)

@script:python@
p1 << r.p1;
p2 << r.p2;
@@

print "* file: %s kmalloc %s return %s" % (p1[0].file,p1[0].line,p2[0].line)
// </smpl>

Signed-off-by: Julia Lawall <julia@diku.dk>
Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: remove dead code
Sage Weil [Fri, 20 Nov 2009 22:44:18 +0000 (14:44 -0800)]
ceph: remove dead code

Left over from mount/auth protocol changes.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: reset msgr backoff during open, not after successful handshake
Sage Weil [Fri, 20 Nov 2009 23:14:15 +0000 (15:14 -0800)]
ceph: reset msgr backoff during open, not after successful handshake

Reset the backoff delay when we reopen the connection, so that the delays
for any initial connection problems are reasonable.  We were resetting only
after a successful handshake, which was of limited utility.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: reset requested max_size after mds reconnect
Sage Weil [Fri, 20 Nov 2009 21:43:45 +0000 (13:43 -0800)]
ceph: reset requested max_size after mds reconnect

The max_size increase request to the MDS can get lost during an MDS
restart and reconnect.  Reset our requested value after the MDS recovers,
so that any blocked writes will re-request a larger max_size upon waking.

Also, explicit wake session caps after the reconnect.  Normally the cap
renewal catches this, but not in the cases where the caps didn't go stale
in the first place, which would leave writers waiting on max_size asleep.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: mount fails immediately on error
Yehuda Sadeh [Fri, 20 Nov 2009 21:59:13 +0000 (13:59 -0800)]
ceph: mount fails immediately on error

Signed-off-by: Yehuda Sadeh <yehuda@newdream.net>
15 years agoceph: decode updated mdsmap format
Sage Weil [Thu, 19 Nov 2009 23:31:50 +0000 (15:31 -0800)]
ceph: decode updated mdsmap format

The mds map now uses the global_id as the 'key' (instead of the addr,
which was a poor choice).

This is protocol change.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: fix debugfs entry, simplify fsid checks
Sage Weil [Thu, 19 Nov 2009 00:50:41 +0000 (16:50 -0800)]
ceph: fix debugfs entry, simplify fsid checks

We may first learn our fsid from any of the mon, osd, or mds maps
(whichever the monitor sends first).  Consolidate checks in a single
helper.  Initialize the client debugfs entry then, since we need the
fsid (and global_id) for the directory name.

Also remove dead mount code.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: small cleanup in hash function
Sage Weil [Thu, 19 Nov 2009 00:50:55 +0000 (16:50 -0800)]
ceph: small cleanup in hash function

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: move mempool creation to ceph_create_client
Sage Weil [Wed, 18 Nov 2009 23:08:44 +0000 (15:08 -0800)]
ceph: move mempool creation to ceph_create_client

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: negotiate authentication protocol; implement AUTH_NONE protocol
Sage Weil [Thu, 19 Nov 2009 00:19:57 +0000 (16:19 -0800)]
ceph: negotiate authentication protocol; implement AUTH_NONE protocol

When we open a monitor session, we send an initial AUTH message listing
the auth protocols we support, our entity name, and (possibly) a previously
assigned global_id.  The monitor chooses a protocol and responds with an
initial message.

Initially implement AUTH_NONE, a dummy protocol that provides no security,
but works within the new framework.  It generates 'authorizers' that are
used when connecting to (mds, osd) services that simply state our entity
name and global_id.

This is a wire protocol change.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: handle errors during osd client init
Sage Weil [Wed, 18 Nov 2009 22:52:18 +0000 (14:52 -0800)]
ceph: handle errors during osd client init

Unwind initializing if we get ENOMEM during client initialization.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: remove unnecessary ceph_con_shutdown
Sage Weil [Wed, 18 Nov 2009 19:27:06 +0000 (11:27 -0800)]
ceph: remove unnecessary ceph_con_shutdown

We require that ceph_con_close be called before we drop the connection,
so this is unneeded.  Just BUG if con->sock != NULL.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: remove bad calls to ceph_con_shutdown
Sage Weil [Wed, 18 Nov 2009 19:22:36 +0000 (11:22 -0800)]
ceph: remove bad calls to ceph_con_shutdown

We want to ceph_con_close when we're done with the connection, before
the ref count reaches 0.  Once it does, do not call ceph_con_shutdown,
as that takes the con mutex and may sleep, and besides that is
unnecessary.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: fix page invalidation deadlock
Sage Weil [Thu, 12 Nov 2009 23:07:24 +0000 (15:07 -0800)]
ceph: fix page invalidation deadlock

We occasionally want to make a best-effort attempt to invalidate cache
pages without fear of blocking.  If this fails, we fall back to an async
invalidate in another thread.

Use invalidate_mapping_pages instead of invalidate_inode_page2, as that
will skip locked pages, and not deadlock.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: build cleanly without CONFIG_DEBUG_FS
Sage Weil [Thu, 12 Nov 2009 23:05:52 +0000 (15:05 -0800)]
ceph: build cleanly without CONFIG_DEBUG_FS

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: pr_info when mds reconnect completes
Sage Weil [Wed, 11 Nov 2009 23:50:12 +0000 (15:50 -0800)]
ceph: pr_info when mds reconnect completes

This helps the user know what's going on during the (involved) reconnect
process.  They already see when the mds fails and reconnect starts.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: initialize i_size/i_rbytes on snapdir
Sage Weil [Wed, 11 Nov 2009 23:22:37 +0000 (15:22 -0800)]
ceph: initialize i_size/i_rbytes on snapdir

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: exclude snapdir from readdir results
Sage Weil [Wed, 11 Nov 2009 23:21:27 +0000 (15:21 -0800)]
ceph: exclude snapdir from readdir results

It was hidden from sync readdir, but not the cached dcache version.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: remove recon_gen logic
Sage Weil [Wed, 11 Nov 2009 00:02:23 +0000 (16:02 -0800)]
ceph: remove recon_gen logic

We don't get an explicit affirmative confirmation that our caps reconnect,
nor do we necessarily want to pay that cost.  So, take all this code out
for now.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: separate banner and connect during handshake into distinct stages
Sage Weil [Tue, 10 Nov 2009 22:34:36 +0000 (14:34 -0800)]
ceph: separate banner and connect during handshake into distinct stages

We need to make sure we only swab the address during the banner once.  So
break process_banner out of process_connect, and clean up the surrounding
code so that these are distinct phases of the handshake.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: do not confuse stale and dead (unreconnected) caps
Sage Weil [Mon, 9 Nov 2009 20:05:48 +0000 (12:05 -0800)]
ceph: do not confuse stale and dead (unreconnected) caps

We were using the cap_gen to track both stale caps (caps that timed out
due to temporarily losing touch with the mds) and dead caps that did not
reconnect after an MDS failure.  Introduce a recon_gen counter to track
reconnections to restarted MDSs and kill dead caps based on that instead.

Rename gen to cap_gen while we're at it to make it more clear which is
which.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: make CRUSH hash function a bucket property
Sage Weil [Sun, 8 Nov 2009 04:18:22 +0000 (20:18 -0800)]
ceph: make CRUSH hash function a bucket property

Make the integer hash function a property of the bucket it is used on.  This
allows us to gracefully add support for new hash functions without starting
from scatch.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: make object hash a pg_pool property
Sage Weil [Sat, 7 Nov 2009 05:55:25 +0000 (21:55 -0800)]
ceph: make object hash a pg_pool property

The object will be hashed to a placement seed (ps) based on the pg_pool's
hash function.  This allows new hashes to be introduced into an existing
object store, or selection of a hash appropriate to the objects that
will be stored in a particular pool.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: use strong hash function for mapping objects to pgs
Sage Weil [Sat, 7 Nov 2009 00:44:05 +0000 (16:44 -0800)]
ceph: use strong hash function for mapping objects to pgs

We were using the (weak) dcache hash function, but it was leaving lower
bits consecutive for consecutive (inode) objects.  We really want to make
the object to pg mapping random and uniform, so use a proper hash function
here.

This is Robert Jenkin's public domain hash function (with some minor
cleanup):
http://burtleburtle.net/bob/hash/evahash.html

This is a protocol revision.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: make CRUSH hash functions non-inline
Sage Weil [Sat, 7 Nov 2009 00:39:26 +0000 (16:39 -0800)]
ceph: make CRUSH hash functions non-inline

These are way to big to be inline.  I missed crush/* when doing the inline
audit for akpm's review.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: clean up 'osd%d down' console msg
Sage Weil [Fri, 6 Nov 2009 21:57:49 +0000 (13:57 -0800)]
ceph: clean up 'osd%d down' console msg

No ceph prefix.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: convert port endianness
Sage Weil [Wed, 4 Nov 2009 19:46:35 +0000 (11:46 -0800)]
ceph: convert port endianness

The port is informational only, but we should make it correct.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: fix sparse endian warning
Sage Weil [Wed, 4 Nov 2009 19:40:05 +0000 (11:40 -0800)]
ceph: fix sparse endian warning

Use the __le macro, even though for -1 it doesn't matter.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: fix endian conversions for ceph_pg
Sage Weil [Wed, 4 Nov 2009 19:39:12 +0000 (11:39 -0800)]
ceph: fix endian conversions for ceph_pg

The endian conversions don't quite work with the old union ceph_pg.  Just
make it a regular struct, and make each field __le.  This is simpler and it
has the added bonus of actually working.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: use fixed endian encoding for ceph_entity_addr
Sage Weil [Tue, 3 Nov 2009 23:17:56 +0000 (15:17 -0800)]
ceph: use fixed endian encoding for ceph_entity_addr

We exchange struct ceph_entity_addr over the wire and store it on disk.
The sockaddr_storage.ss_family field, however, is host endianness.  So,
fix ss_family endianness to big endian when sending/receiving over the
wire.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: init/destroy bdi in client create/destroy helpers
Sage Weil [Mon, 2 Nov 2009 17:32:47 +0000 (09:32 -0800)]
ceph: init/destroy bdi in client create/destroy helpers

This keeps bdi setup/teardown in line with client life cycle.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agocrush: always return a value from crush_bucket_choose
Sage Weil [Mon, 2 Nov 2009 01:53:24 +0000 (17:53 -0800)]
crush: always return a value from crush_bucket_choose

Even when we encounter a corrupt bucket.  We still BUG().  This fixes
the warning

fs/ceph/crush/mapper.c: In function 'crush_choose':
fs/ceph/crush/mapper.c:352: warning: control may reach end of non-void function
'crush_bucket_choose' being inlined

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: fix uninitialized err variable
Sage Weil [Mon, 2 Nov 2009 01:51:15 +0000 (17:51 -0800)]
ceph: fix uninitialized err variable

Fixes warning
fs/ceph/xattr.c: In function '__build_xattrs':
fs/ceph/xattr.c:353: warning: 'err' may be used uninitialized in this function

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: fix intra strip unit length calculation
Noah Watkins [Fri, 30 Oct 2009 19:57:30 +0000 (12:57 -0700)]
ceph: fix intra strip unit length calculation

Commit 645a102581b3639836b17d147c35d574fd6e8267 fixes calculation of object
offset for layouts with multiple stripes per object. This updates the
calculation of the length written to take into account multiple stripes per
object.

Signed-off-by: Noah Watkins <noah@noahdesu.com>
Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: fix object striping calculation for non-default striping schemes
Sage Weil [Wed, 28 Oct 2009 22:15:05 +0000 (15:15 -0700)]
ceph: fix object striping calculation for non-default striping schemes

We were incorrectly calculationing of object offset.  If we have multiple
stripe units per object, we need to shift to the start of the current
su in addition to the offset within the su.

Also rename bno to ono (object number) to avoid some variable naming
confusion.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: correct comment to match striping calculation
Sage Weil [Wed, 28 Oct 2009 21:57:25 +0000 (14:57 -0700)]
ceph: correct comment to match striping calculation

The object extent offset is the file offset _modulo_ the stripe unit.
The code was correct, the comment was wrong.

Reported-by: Noah Watkins <jayhawk@soe.ucsc.edu>
Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: remove redundant use of le32_to_cpu
Noah Watkins [Wed, 28 Oct 2009 21:04:48 +0000 (14:04 -0700)]
ceph: remove redundant use of le32_to_cpu

Using stripe unit size calculated and saved on the stack to avoid
a redundant call to le32_to_cpu.

Signed-off-by: Noah Watkins <noah@noahdesu.com>
Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: replace list_entry with container_of
Noah Watkins [Wed, 28 Oct 2009 18:54:49 +0000 (11:54 -0700)]
ceph: replace list_entry with container_of

Usage of non-list.h list_entry function for container_of
functionality replaced with direct use of container_of.

Signed-off-by: Noah Watkins <noah@noahdesu.com>
Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: allocate and parse mount args before client instance
Sage Weil [Tue, 27 Oct 2009 18:50:50 +0000 (11:50 -0700)]
ceph: allocate and parse mount args before client instance

This simplifies much of the error handling during mount.  It also means
that we have the mount args before client creation, and we can initialize
based on those options.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: fix, clean up string mount arg parsing
Sage Weil [Tue, 27 Oct 2009 17:19:28 +0000 (10:19 -0700)]
ceph: fix, clean up string mount arg parsing

Clearly demark int and string argument options, and do not try to convert
string arguments to ints.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: silence uninitialized variable warning
Sage Weil [Tue, 27 Oct 2009 05:06:22 +0000 (22:06 -0700)]
ceph: silence uninitialized variable warning

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: reduce parse_mount_args stack usage
Sage Weil [Tue, 27 Oct 2009 05:07:53 +0000 (22:07 -0700)]
ceph: reduce parse_mount_args stack usage

Since we've increased the max mon count, we shouldn't put the addr array
on the parse_mount_args stack.  Put it on the heap instead.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: remove small mon addr limit; use CEPH_MAX_MON where appropriate
Sage Weil [Thu, 22 Oct 2009 17:53:02 +0000 (10:53 -0700)]
ceph: remove small mon addr limit; use CEPH_MAX_MON where appropriate

Get rid of separate max mon limit; use the system limit instead.  This
allows mounts when there are lots of mon addrs provided by mount.ceph (as
with a host with lots of A/AAAA records).

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: move directory size logic to ceph_getattr
Sage Weil [Wed, 21 Oct 2009 18:21:49 +0000 (11:21 -0700)]
ceph: move directory size logic to ceph_getattr

We can't fill i_size with rbytes at the fill_file_size stage without
adding additional checks for directories.  Notably, we want st_blocks
to remain 0 on directories so that 'du' still works.

Fill in i_blocks, i_size specially in ceph_getattr instead.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: v0.17 of client
Sage Weil [Mon, 19 Oct 2009 23:17:31 +0000 (16:17 -0700)]
ceph: v0.17 of client

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: include preferred osd in placement seed
Sage Weil [Mon, 19 Oct 2009 18:41:51 +0000 (11:41 -0700)]
ceph: include preferred osd in placement seed

Mix the preferred osd (if any) into the placement seed that is fed into
the CRUSH object placement calculation.  This prevents all the placement
pgs from peering with the same osds.

Rev the osd client protocol with this change.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: enable readahead
Sage Weil [Fri, 16 Oct 2009 21:44:35 +0000 (14:44 -0700)]
ceph: enable readahead

Initialized bdi->ra_pages to enable readahead.  Use 512KB default.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: move dirty caps code around
Sage Weil [Fri, 16 Oct 2009 01:13:53 +0000 (18:13 -0700)]
ceph: move dirty caps code around

Cleanup only.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: warn on allocation from msgpool with larger front_len
Sage Weil [Thu, 15 Oct 2009 00:36:07 +0000 (17:36 -0700)]
ceph: warn on allocation from msgpool with larger front_len

Pass the front_len we need when pulling a message off a msgpool,
and WARN if it is greater than the pool's size.  Then try to
allocate a new message (to continue without failing).

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: correct subscribe_ack msgpool payload size
Sage Weil [Thu, 15 Oct 2009 00:26:40 +0000 (17:26 -0700)]
ceph: correct subscribe_ack msgpool payload size

Defined a struct for the SUBSCRIBE_ACK, and use that to size
the msgpool.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: flush dirty caps via the cap_dirty list
Sage Weil [Wed, 14 Oct 2009 21:27:38 +0000 (14:27 -0700)]
ceph: flush dirty caps via the cap_dirty list

Previously we were flushing dirty caps by passing an extra flag
when traversing the delayed caps list.  Besides being a bit ugly,
that can also miss caps that are dirty but didn't result in a
cap requeue: notably, mark_caps_dirty().

Separate the flushing into a separate helper, and traverse the
cap_dirty list.

This also brings i_dirty_item in line with i_dirty_caps: we are
on the list IFF caps != 0.  We carry an inode ref IFF
dirty_caps|flushing_caps != 0.

Lose the unused return value from __ceph_mark_caps_dirty().

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: move generic flushing code into helper
Sage Weil [Wed, 14 Oct 2009 21:24:19 +0000 (14:24 -0700)]
ceph: move generic flushing code into helper

Both callers of __mark_caps_flushing() do the same work; move it
into the helper.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: initialize sb->s_bdi, bdi_unregister after kill_anon_super
Sage Weil [Wed, 14 Oct 2009 21:09:07 +0000 (14:09 -0700)]
ceph: initialize sb->s_bdi, bdi_unregister after kill_anon_super

Writeback doesn't work without the bdi set, and writeback on
umount doesn't work if we unregister the bdi too early.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: convert encode/decode macros to inlines
Sage Weil [Wed, 14 Oct 2009 16:59:09 +0000 (09:59 -0700)]
ceph: convert encode/decode macros to inlines

This avoids the fugly pass by reference and makes the code a bit easier
to read.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: add version field to message header
Sage Weil [Tue, 13 Oct 2009 19:55:26 +0000 (12:55 -0700)]
ceph: add version field to message header

This makes it easier for individual message types to indicate
their particular encoding, and make future changes backward
compatible.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: remove unused CEPH_MSG_{OSD,MDS}_GETMAP
Sage Weil [Sat, 10 Oct 2009 04:52:34 +0000 (21:52 -0700)]
ceph: remove unused CEPH_MSG_{OSD,MDS}_GETMAP

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: ignore trailing data in monamp
Sage Weil [Mon, 12 Oct 2009 17:28:13 +0000 (10:28 -0700)]
ceph: ignore trailing data in monamp

This lets us extend the format more easily.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: add file layout validation
Sage Weil [Fri, 9 Oct 2009 23:38:45 +0000 (16:38 -0700)]
ceph: add file layout validation

This tracks updates to code shared with userspace.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: update to mon client protocol v15
Sage Weil [Fri, 9 Oct 2009 23:36:34 +0000 (16:36 -0700)]
ceph: update to mon client protocol v15

The mon request headers now include session_mon information that must
be properly initialized.

Signed-off-by: Sage Weil <sage@newdream.net>