mbox series

[v3,0/5] virtiofsd xattr name mappings

Message ID 20201014180209.49299-1-dgilbert@redhat.com
Headers show
Series virtiofsd xattr name mappings | expand

Message

Dr. David Alan Gilbert Oct. 14, 2020, 6:02 p.m. UTC
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>

This is a 3rd cut of an xattr name mapping option for virtiofsd.
It allows the user of virtiofsd to define a fairly flexible mapping
from the view of the xattr names the host fs has and the ones that the
guest sees.

  The hope is this allows things like:
    a) Different selinux attributes on host/guest
    b) separation of trusted. attributes that clash on overlayfs
    c) support for privileged xattr's in guests running with an
       unprivileged virtiofsd.

There's no apparent standard for this kind of mapping, so I made
it flexible by specifying a mapping rule in the option.

Prefix's can be added (selectively or globally), xattr's can be
dropped in either direction or passed through.

The major change for v3 is the addition of a 'map' simple syntax
that should cover a lot of the simple cases without people needing
to use the more complex rule syntax.  There's also some cleanups
basedon reviews by Christophe.

Dave


Dr. David Alan Gilbert (5):
  tools/virtiofsd: xattr name mappings: Add option
  tools/virtiofsd: xattr name mappings: Map client xattr names
  tools/virtiofsd: xattr name mappings: Map server xattr names
  tools/virtiofsd: xattr name mapping examples
  tools/virtiofsd: xattr name mappings: Simple 'map'

 docs/tools/virtiofsd.rst         | 122 ++++++++
 tools/virtiofsd/passthrough_ll.c | 476 ++++++++++++++++++++++++++++++-
 2 files changed, 595 insertions(+), 3 deletions(-)

Comments

Stefan Hajnoczi Oct. 20, 2020, 9:56 a.m. UTC | #1
On Wed, Oct 14, 2020 at 07:02:08PM +0100, Dr. David Alan Gilbert (git) wrote:
> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> 
> Add a few examples of xattrmaps to the documentation.
> 
> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> ---
>  docs/tools/virtiofsd.rst | 50 ++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 50 insertions(+)

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Stefan Hajnoczi Oct. 20, 2020, 10:09 a.m. UTC | #2
On Wed, Oct 14, 2020 at 07:02:09PM +0100, Dr. David Alan Gilbert (git) wrote:
> +static XattrMapEntry *parse_xattrmap_map(const char *rule,
> +                                         XattrMapEntry *map,
> +                                         size_t *nentries)
> +{
> +    char sep = *rule++;
> +    const char *tmp;
> +    char *key;
> +    char *prefix;
> +    XattrMapEntry tmp_entry;
> +
> +    /* At start of 'key' field */
> +    tmp = strchr(rule, sep);

Missing sep == '\0' check. The strchr(3) man page says:

  The terminating null byte is considered part of the string, so that if
  c is specified as '\0', these functions return a pointer to the
  terminator.

So the code in this patch will eventually access beyond the end of the
string:

  rule = tmp + 1; <-- tmp is already at the NUL terminator
Dr. David Alan Gilbert Oct. 20, 2020, 11:35 a.m. UTC | #3
* Stefan Hajnoczi (stefanha@redhat.com) wrote:
> On Wed, Oct 14, 2020 at 07:02:09PM +0100, Dr. David Alan Gilbert (git) wrote:
> > +static XattrMapEntry *parse_xattrmap_map(const char *rule,
> > +                                         XattrMapEntry *map,
> > +                                         size_t *nentries)
> > +{
> > +    char sep = *rule++;
> > +    const char *tmp;
> > +    char *key;
> > +    char *prefix;
> > +    XattrMapEntry tmp_entry;
> > +
> > +    /* At start of 'key' field */
> > +    tmp = strchr(rule, sep);
> 
> Missing sep == '\0' check. The strchr(3) man page says:
> 
>   The terminating null byte is considered part of the string, so that if
>   c is specified as '\0', these functions return a pointer to the
>   terminator.
> 
> So the code in this patch will eventually access beyond the end of the
> string:
> 
>   rule = tmp + 1; <-- tmp is already at the NUL terminator

Ah yes, I've got the check in the main routine but not in the _map case;
OK I'll fix that up.

Dave
Vivek Goyal Oct. 20, 2020, 2:04 p.m. UTC | #4
On Wed, Oct 14, 2020 at 07:02:05PM +0100, Dr. David Alan Gilbert (git) wrote:
> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> 
> Add an option to define mappings of xattr names so that
> the client and server filesystems see different views.
> This can be used to have different SELinux mappings as
> seen by the guest, to run the virtiofsd with less privileges
> (e.g. in a case where it can't set trusted/system/security
> xattrs but you want the guest to be able to), or to isolate
> multiple users of the same name; e.g. trusted attributes
> used by stacking overlayfs.
> 
> A mapping engine is used wit 3 simple rules; the rules can
> be combined to allow most useful mapping scenarios.
> The ruleset is defined by -o xattrmap='rules...'.
> 
> This patch doesn't use the rule maps yet.
> 
> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> ---
>  docs/tools/virtiofsd.rst         |  54 ++++++++++
>  tools/virtiofsd/passthrough_ll.c | 180 +++++++++++++++++++++++++++++++
>  2 files changed, 234 insertions(+)
> 
> diff --git a/docs/tools/virtiofsd.rst b/docs/tools/virtiofsd.rst
> index 7ecee49834..a3a120da2f 100644
> --- a/docs/tools/virtiofsd.rst
> +++ b/docs/tools/virtiofsd.rst
> @@ -109,6 +109,60 @@ Options
>    timeout.  ``always`` sets a long cache lifetime at the expense of coherency.
>    The default is ``auto``.
>  
> +xattr-mapping
> +-------------
> +
> +By default the name of xattr's used by the client are passed through to the server
> +file system.  This can be a problem where either those xattr names are used
> +by something on the server (e.g. selinux client/server confusion) or if the
> +virtiofsd is running in a container with restricted privileges where it cannot
> +access some attributes.
> +
> +A mapping of xattr names can be made using -o xattrmap=mapping where the ``mapping``
> +string consists of a series of rules.
> +
> +The first matching rule terminates the mapping.
> +
> +Each rule consists of a number of fields separated with a separator that is the
> +first non-white space character in the rule.  This separator must then be used
> +for the whole rule.

Where is it useful to have a separator other than ":".


> +White space may be added before and after each rule.
> +Using ':' as the separator a rule is of the form:
> +
> +``:type:scope:key:prepend:``
> +
> +**type** is one of:

I am not sure I understand the syntax explanation. So I will ask
some basic questions.

> +
> +- 'prefix' - If 'key' matches the client then the 'prepend'

What do you mean by "If key matches the client"?

> +  is added before the name is passed to the server.

Its basically the sever which is processing these rules. I guess
you have written this sentence from the perspective of "rule engine"
and it passes the name to file server (despite the fact that rule
engine itself is part of file server).

> +  For a server case, the prepend is tested and stripped
> +  if matching.

Is this about file server removing the "prepend" from xattr
name before it is sent back to client?

> +
> +- 'ok' - The attribute name is OK and passed through to
> +  the server unchanged.
> +
> +- 'bad' - If a client tries to use this name it's
> +  denied using EPERM; when the server passes an attribute
> +  name matching it's hidden.
> +
> +**scope** is:
> +
> +- 'client' - match 'key' against a xattr name from the client for
> +             setxattr/getxattr/removexattr
> +- 'server' - match 'prepend' against a xattr name from the server
> +             for listxattr
> +- 'all' - can be used to match both cases.

So we need this only if we don't want to do
xattr mapping bidirectionally. IOW, server will not allow
getxattr "user.foo" but will allow listing "user.foo"?

/me is wondering what's the use case of this.

> +
> +**key** is a string tested as a prefix on an attribute name originating
> +on the client.  It maybe empty in which case a 'client' rule
> +will always match on client names.
> +
> +**prepend** is a string tested as a prefix on an attribute name originating
> +on the server, and used as a new prefix.

What does it mean that "prepend" is tested as a prefix" and "used as a
new prefix". It is used as a new prefix for client rule, right? Atleast
documentation does not make it clear.

> It may be empty
> +in which case a 'server' rule will always match on all names from
> +the server.
> +
> +
>  Examples
>  --------

Can we give some examples in this patch so that it is easy to understand
what these rules can allow.

Thanks
Vivek

>  
> diff --git a/tools/virtiofsd/passthrough_ll.c b/tools/virtiofsd/passthrough_ll.c
> index ff53df4451..f5a33014f9 100644
> --- a/tools/virtiofsd/passthrough_ll.c
> +++ b/tools/virtiofsd/passthrough_ll.c
> @@ -64,6 +64,7 @@
>  #include <syslog.h>
>  #include <unistd.h>
>  
> +#include "qemu/cutils.h"
>  #include "passthrough_helpers.h"
>  #include "passthrough_seccomp.h"
>  
> @@ -137,6 +138,12 @@ enum {
>      CACHE_ALWAYS,
>  };
>  
> +typedef struct xattr_map_entry {
> +    char *key;
> +    char *prepend;
> +    unsigned int flags;
> +} XattrMapEntry;
> +
>  struct lo_data {
>      pthread_mutex_t mutex;
>      int debug;
> @@ -144,6 +151,7 @@ struct lo_data {
>      int flock;
>      int posix_lock;
>      int xattr;
> +    char *xattrmap;
>      char *source;
>      char *modcaps;
>      double timeout;
> @@ -157,6 +165,7 @@ struct lo_data {
>      struct lo_map ino_map; /* protected by lo->mutex */
>      struct lo_map dirp_map; /* protected by lo->mutex */
>      struct lo_map fd_map; /* protected by lo->mutex */
> +    XattrMapEntry *xattr_map_list;
>  
>      /* An O_PATH file descriptor to /proc/self/fd/ */
>      int proc_self_fd;
> @@ -172,6 +181,7 @@ static const struct fuse_opt lo_opts[] = {
>      { "no_posix_lock", offsetof(struct lo_data, posix_lock), 0 },
>      { "xattr", offsetof(struct lo_data, xattr), 1 },
>      { "no_xattr", offsetof(struct lo_data, xattr), 0 },
> +    { "xattrmap=%s", offsetof(struct lo_data, xattrmap), 0 },
>      { "modcaps=%s", offsetof(struct lo_data, modcaps), 0 },
>      { "timeout=%lf", offsetof(struct lo_data, timeout), 0 },
>      { "timeout=", offsetof(struct lo_data, timeout_set), 1 },
> @@ -2010,6 +2020,169 @@ static void lo_flock(fuse_req_t req, fuse_ino_t ino, struct fuse_file_info *fi,
>      fuse_reply_err(req, res == -1 ? errno : 0);
>  }
>  
> +/*
> + * Exit; process attribute unmodified if matched.
> + * An empty key applies to all.
> + */
> +#define XATTR_MAP_FLAG_END_OK  (1 <<  0)
> +/*
> + * The attribute is unwanted;
> + * EPERM on write hidden on read.
> + */
> +#define XATTR_MAP_FLAG_END_BAD (1 <<  1)
> +/*
> + * For attr that start with 'key' prepend 'prepend'
> + * 'key' maybe empty to prepend for all attrs
> + * key is defined from set/remove point of view.
> + * Automatically reversed on read
> + */
> +#define XATTR_MAP_FLAG_PREFIX  (1 <<  2)
> +/* Apply rule to get/set/remove */
> +#define XATTR_MAP_FLAG_CLIENT  (1 << 16)
> +/* Apply rule to list */
> +#define XATTR_MAP_FLAG_SERVER  (1 << 17)
> +/* Apply rule to all */
> +#define XATTR_MAP_FLAG_ALL   (XATTR_MAP_FLAG_SERVER | XATTR_MAP_FLAG_CLIENT)
> +
> +/* Last rule in the XATTR_MAP */
> +#define XATTR_MAP_FLAG_LAST    (1 << 30)
> +
> +static XattrMapEntry *add_xattrmap_entry(XattrMapEntry *orig_map,
> +                                         size_t *nentries,
> +                                         const XattrMapEntry *new_entry)
> +{
> +    XattrMapEntry *res = g_realloc_n(orig_map, ++*nentries,
> +                                     sizeof(XattrMapEntry));
> +    res[*nentries - 1] = *new_entry;
> +
> +    return res;
> +}
> +
> +static void free_xattrmap(XattrMapEntry *map)
> +{
> +    XattrMapEntry *curr = map;
> +
> +    if (!map) {
> +        return;
> +    };
> +
> +    do {
> +        g_free(curr->key);
> +        g_free(curr->prepend);
> +    } while (!(curr++->flags & XATTR_MAP_FLAG_LAST));
> +
> +    g_free(map);
> +}
> +
> +static XattrMapEntry *parse_xattrmap(struct lo_data *lo)
> +{
> +    XattrMapEntry *res = NULL;
> +    XattrMapEntry tmp_entry;
> +    size_t nentries = 0;
> +    const char *map = lo->xattrmap;
> +    const char *tmp;
> +
> +    while (*map) {
> +        char sep;
> +
> +        if (isspace(*map)) {
> +            map++;
> +            continue;
> +        }
> +        /* The separator is the first non-space of the rule */
> +        sep = *map++;
> +        if (!sep) {
> +            break;
> +        }
> +
> +        /* Start of 'type' */
> +        if (strstart(map, "prefix", &map)) {
> +            tmp_entry.flags |= XATTR_MAP_FLAG_PREFIX;
> +        } else if (strstart(map, "ok", &map)) {
> +            tmp_entry.flags |= XATTR_MAP_FLAG_END_OK;
> +        } else if (strstart(map, "bad", &map)) {
> +            tmp_entry.flags |= XATTR_MAP_FLAG_END_BAD;
> +        } else {
> +            fuse_log(FUSE_LOG_ERR,
> +                     "%s: Unexpected type;"
> +                     "Expecting 'prefix', 'ok', or 'bad' in rule %zu\n",
> +                     __func__, nentries);
> +            exit(1);
> +        }
> +
> +        if (*map++ != sep) {
> +            fuse_log(FUSE_LOG_ERR,
> +                     "%s: Missing '%c' at end of type field of rule %zu\n",
> +                     __func__, sep, nentries);
> +            exit(1);
> +        }
> +
> +        /* Start of 'scope' */
> +        if (strstart(map, "client", &map)) {
> +            tmp_entry.flags |= XATTR_MAP_FLAG_CLIENT;
> +        } else if (strstart(map, "server", &map)) {
> +            tmp_entry.flags |= XATTR_MAP_FLAG_SERVER;
> +        } else if (strstart(map, "all", &map)) {
> +            tmp_entry.flags |= XATTR_MAP_FLAG_ALL;
> +        } else {
> +            fuse_log(FUSE_LOG_ERR,
> +                     "%s: Unexpected scope;"
> +                     " Expecting 'client', 'server', or 'all', in rule %zu\n",
> +                     __func__, nentries);
> +            exit(1);
> +        }
> +
> +        if (*map++ != sep) {
> +            fuse_log(FUSE_LOG_ERR,
> +                     "%s: Expecting '%c' found '%c'"
> +                     " after scope in rule %zu\n",
> +                     __func__, sep, *map, nentries);
> +            exit(1);
> +        }
> +
> +        /* At start of 'key' field */
> +        tmp = strchr(map, sep);
> +        if (!tmp) {
> +            fuse_log(FUSE_LOG_ERR,
> +                     "%s: Missing '%c' at end of key field of rule %zu",
> +                     __func__, sep, nentries);
> +            exit(1);
> +        }
> +        tmp_entry.key = g_strndup(map, tmp - map);
> +        map = tmp + 1;
> +
> +        /* At start of 'prepend' field */
> +        tmp = strchr(map, sep);
> +        if (!tmp) {
> +            fuse_log(FUSE_LOG_ERR,
> +                     "%s: Missing '%c' at end of prepend field of rule %zu",
> +                     __func__, sep, nentries);
> +            exit(1);
> +        }
> +        tmp_entry.prepend = g_strndup(map, tmp - map);
> +        map = tmp + 1;
> +
> +        lo->xattr_map_list = add_xattrmap_entry(lo->xattr_map_list, &nentries,
> +                                                &tmp_entry);
> +        /* End of rule - go around again for another rule */
> +    }
> +
> +    if (!nentries) {
> +        fuse_log(FUSE_LOG_ERR, "Empty xattr map\n");
> +        exit(1);
> +    }
> +
> +    /* Add a terminator to error in cases the user hasn't specified */
> +    tmp_entry.flags = XATTR_MAP_FLAG_ALL | XATTR_MAP_FLAG_END_BAD |
> +                      XATTR_MAP_FLAG_LAST;
> +    tmp_entry.key = g_strdup("");
> +    tmp_entry.prepend = g_strdup("");
> +    lo->xattr_map_list = add_xattrmap_entry(lo->xattr_map_list, &nentries,
> +                                            &tmp_entry);
> +
> +    return res;
> +}
> +
>  static void lo_getxattr(fuse_req_t req, fuse_ino_t ino, const char *name,
>                          size_t size)
>  {
> @@ -2806,6 +2979,8 @@ static void fuse_lo_data_cleanup(struct lo_data *lo)
>          close(lo->root.fd);
>      }
>  
> +    free(lo->xattrmap);
> +    free_xattrmap(lo->xattr_map_list);
>      free(lo->source);
>  }
>  
> @@ -2906,6 +3081,11 @@ int main(int argc, char *argv[])
>      } else {
>          lo.source = strdup("/");
>      }
> +
> +    if (lo.xattrmap) {
> +        lo.xattr_map_list = parse_xattrmap(&lo);
> +    }
> +
>      if (!lo.timeout_set) {
>          switch (lo.cache) {
>          case CACHE_NONE:
> -- 
> 2.28.0
>
Dr. David Alan Gilbert Oct. 20, 2020, 3:34 p.m. UTC | #5
* Vivek Goyal (vgoyal@redhat.com) wrote:
> On Wed, Oct 14, 2020 at 07:02:08PM +0100, Dr. David Alan Gilbert (git) wrote:
> > From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> > 
> > Add a few examples of xattrmaps to the documentation.
> > 
> > Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> > ---
> >  docs/tools/virtiofsd.rst | 50 ++++++++++++++++++++++++++++++++++++++++
> >  1 file changed, 50 insertions(+)
> > 
> > diff --git a/docs/tools/virtiofsd.rst b/docs/tools/virtiofsd.rst
> > index a3a120da2f..5cb64612ed 100644
> > --- a/docs/tools/virtiofsd.rst
> > +++ b/docs/tools/virtiofsd.rst
> > @@ -163,6 +163,56 @@ in which case a 'server' rule will always match on all names from
> >  the server.
> >  
> >  
> > +xattr-mapping Examples
> > +----------------------
> > +
> > +1) Prefix all attributes with 'user.virtiofs.'
> > +
> > +::
> > +
> > +-o xattrmap=":prefix:all::user.virtiofs.::bad:all:::"
> > +
> 
> Staring at rule.
> 
> ":bad:all:::"
> 
> and trying to map this to ":type:scope:key:prepend:"
> 
> type==bad
> scope==all
> key==""
> prepend==""

Correct.

> > +
> > +This uses two rules, using : as the field separator;
> > +the first rule prefixes and strips 'user.virtiofs.',
> > +the second rule hides any non-prefixed attributes that
> > +the host set.
> 
> What is non-prefixed xattr in this context. If client sends
> "security.selinux", is prefixed or non-prefixed.

Note that anything originating at the client (i.e. starting with "")
will get caught by the first rule and be prefixed with 'user.virtiofs.'
This second rule will only be triggered by an xattr name coming
from the server (i.e a listxattr) for a name that doesn't begin
with user.virtiofs. (i.e. didn't match the 1st rule for a server xattr).
This makes sure that only guest created xattr's (that were set and
had a prefix added) are then visible to the guest.

> Documentation in first patch said.
> 
> +- 'bad' - If a client tries to use this name it's
> +  denied using EPERM; when the server passes an attribute
> +  name matching it's hidden.
> 
> I am not sure which xattr name will be blocked if key=="" and prepend==""

All of them; they're still matched at the beginning; as the first
patch says 'It maybe empty in which case a 'client' rule will always
match on client names'

> > +2) Prefix 'trusted.' attributes, allow others through
> > +
> > +::
> > +
> > +   "/prefix/all/trusted./user.virtiofs./
> > +    /bad/server//trusted./
> > +    /bad/client/user.virtiofs.//
> > +    /ok/all///"
> > +
> > +
> > +Here there are four rules, using / as the field
> > +separator, and also demonstrating that new lines can
> > +be included between rules.
> > +The first rule is the prefixing of 'trusted.' and
> > +stripping of 'user.virtiofs.'.
> 
> So this is bidrectional rule, right. For setxattr(), "trusted."
> will be replaced with "user.virtiofs" and for listxattr(),
> server will replace user.virtiofs with trusted. ?

prefixed not replaced; so it'll turn "trusted." into
"user.virtiofs.trusted." and strip it back off for listxattr.

> > +The second rule hides unprefixed 'trusted.' attributes
> > +on the host.
> 
> If host has "trusted.*", we are not hiding it and as per first
> rule we are converting it to "user.virtiofs.trusted.*", right?
> So why this second rule is needed.

No, the first rule will only prefix strings provided by the guest
and strip strings provided by the server. This rule hides
existing server 'trusted.' xattrs - so if the guest sets
trusted.foo it's not confused by also seeing a server trusted.foo

> > +The third rule stops a guest from explicitly setting
> > +the 'user.viritofs.' path directly.
> > +Finally, the fourth rule lets all remaining attributes
> > +through.
> 
> So If I don't specify third rule, and client does
> setxattr(user.virtiofs.*), it will simply be a passthrough?

Right; and that's dangerous, because a non-privileged guest
process can set a user. xattr; so a non-priv guest process could
set user.virtiofs.trusted.foo and then it would get read back
and be used as trusted.foo that has an impact on priviliged processes.

Dave

> Thanks
> Vivek
> 
> > +
> > +3) Hide 'security.' attributes, and allow everything else
> > +
> > +::
> > +
> > +    "/bad/all/security./security./
> > +     /ok/all///'
> > +
> > +The first rule combines what could be separate client and server
> > +rules into a single 'all' rule, matching 'security.' in either
> > +client arguments or lists returned from the host.  This stops
> > +the client seeing any 'security.' attributes on the server and
> > +stops it setting any.
> > +
> >  Examples
> >  --------
> >  
> > -- 
> > 2.28.0
> >
Vivek Goyal Oct. 20, 2020, 5:56 p.m. UTC | #6
On Tue, Oct 20, 2020 at 04:34:43PM +0100, Dr. David Alan Gilbert wrote:
> * Vivek Goyal (vgoyal@redhat.com) wrote:
> > On Wed, Oct 14, 2020 at 07:02:08PM +0100, Dr. David Alan Gilbert (git) wrote:
> > > From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> > > 
> > > Add a few examples of xattrmaps to the documentation.
> > > 
> > > Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> > > ---
> > >  docs/tools/virtiofsd.rst | 50 ++++++++++++++++++++++++++++++++++++++++
> > >  1 file changed, 50 insertions(+)
> > > 
> > > diff --git a/docs/tools/virtiofsd.rst b/docs/tools/virtiofsd.rst
> > > index a3a120da2f..5cb64612ed 100644
> > > --- a/docs/tools/virtiofsd.rst
> > > +++ b/docs/tools/virtiofsd.rst
> > > @@ -163,6 +163,56 @@ in which case a 'server' rule will always match on all names from
> > >  the server.
> > >  
> > >  
> > > +xattr-mapping Examples
> > > +----------------------
> > > +
> > > +1) Prefix all attributes with 'user.virtiofs.'
> > > +
> > > +::
> > > +
> > > +-o xattrmap=":prefix:all::user.virtiofs.::bad:all:::"
> > > +
> > 
> > Staring at rule.
> > 
> > ":bad:all:::"
> > 
> > and trying to map this to ":type:scope:key:prepend:"
> > 
> > type==bad
> > scope==all
> > key==""
> > prepend==""
> 
> Correct.
> 
> > > +
> > > +This uses two rules, using : as the field separator;
> > > +the first rule prefixes and strips 'user.virtiofs.',
> > > +the second rule hides any non-prefixed attributes that
> > > +the host set.
> > 
> > What is non-prefixed xattr in this context. If client sends
> > "security.selinux", is prefixed or non-prefixed.
> 
> Note that anything originating at the client (i.e. starting with "")
> will get caught by the first rule and be prefixed with 'user.virtiofs.'

So first rule is.

:prefix:all::user.virtiofs.:

scope="all"
key==""
prepend="user.virtiofs."

Given scope, rule applies to both client and server. Given key is "",
any income setxattr() request will be prefixed with "user.virtiofs".
And how does this rule work for server (listxattr()). It will match
prepend "user.virtiofs" and if that matches it will be stripped?


> This second rule will only be triggered by an xattr name coming
> from the server (i.e a listxattr) for a name that doesn't begin
> with user.virtiofs. (i.e. didn't match the 1st rule for a server xattr).
> This makes sure that only guest created xattr's (that were set and
> had a prefix added) are then visible to the guest.

Ok, so if an xattr does not match first rule, then second rule says
key="", prepend="" and that will match all xattrs. So anything which
is not caught by first rule, will be caught by second rule and
either rejected or filtered out.

> 
> > Documentation in first patch said.
> > 
> > +- 'bad' - If a client tries to use this name it's
> > +  denied using EPERM; when the server passes an attribute
> > +  name matching it's hidden.
> > 
> > I am not sure which xattr name will be blocked if key=="" and prepend==""
> 
> All of them; they're still matched at the beginning; as the first
> patch says 'It maybe empty in which case a 'client' rule will always
> match on client names'

Ok.

> 
> > > +2) Prefix 'trusted.' attributes, allow others through
> > > +
> > > +::
> > > +
> > > +   "/prefix/all/trusted./user.virtiofs./
> > > +    /bad/server//trusted./
> > > +    /bad/client/user.virtiofs.//
> > > +    /ok/all///"
> > > +
> > > +
> > > +Here there are four rules, using / as the field
> > > +separator, and also demonstrating that new lines can
> > > +be included between rules.
> > > +The first rule is the prefixing of 'trusted.' and
> > > +stripping of 'user.virtiofs.'.
> > 
> > So this is bidrectional rule, right. For setxattr(), "trusted."
> > will be replaced with "user.virtiofs" and for listxattr(),
> > server will replace user.virtiofs with trusted. ?
> 
> prefixed not replaced; so it'll turn "trusted." into
> "user.virtiofs.trusted." and strip it back off for listxattr.

Ok. Got it. I am wondering how will I specify these rules so that
they work in nested configuration. Say I have L0 host, L1 guest and
L2 guest. Say virtiofsd0 is running on L0 and virtiofsd1 is running
on L1. 

I am wondering how will I specify the rules on virtiofsd0 and virtiofsd1
so that it works. Will it be same or rules are level dependent.

> 
> > > +The second rule hides unprefixed 'trusted.' attributes
> > > +on the host.
> > 
> > If host has "trusted.*", we are not hiding it and as per first
> > rule we are converting it to "user.virtiofs.trusted.*", right?
> > So why this second rule is needed.
> 
> No, the first rule will only prefix strings provided by the guest
> and strip strings provided by the server. This rule hides
> existing server 'trusted.' xattrs - so if the guest sets
> trusted.foo it's not confused by also seeing a server trusted.foo
> 
> > > +The third rule stops a guest from explicitly setting
> > > +the 'user.viritofs.' path directly.
> > > +Finally, the fourth rule lets all remaining attributes
> > > +through.
> > 
> > So If I don't specify third rule, and client does
> > setxattr(user.virtiofs.*), it will simply be a passthrough?
> 
> Right; and that's dangerous, because a non-privileged guest
> process can set a user. xattr; so a non-priv guest process could
> set user.virtiofs.trusted.foo and then it would get read back
> and be used as trusted.foo that has an impact on priviliged processes.

Right. We don't want unpriviliged process to be able to setup
user.virtiofs.trusted.*. But that's what precisely happen in
a nested configuration.

In above example, L2 will set trusted.foo, virtiofsd1 wil convert it
to user.virtiofs.trusted.foo and virtiofsd0 will reject it, breaking
the nested virtiofs.

Thanks
Vivek
Dr. David Alan Gilbert Oct. 20, 2020, 7:02 p.m. UTC | #7
* Vivek Goyal (vgoyal@redhat.com) wrote:
> On Tue, Oct 20, 2020 at 04:34:43PM +0100, Dr. David Alan Gilbert wrote:
> > * Vivek Goyal (vgoyal@redhat.com) wrote:
> > > On Wed, Oct 14, 2020 at 07:02:08PM +0100, Dr. David Alan Gilbert (git) wrote:
> > > > From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> > > > 
> > > > Add a few examples of xattrmaps to the documentation.
> > > > 
> > > > Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> > > > ---
> > > >  docs/tools/virtiofsd.rst | 50 ++++++++++++++++++++++++++++++++++++++++
> > > >  1 file changed, 50 insertions(+)
> > > > 
> > > > diff --git a/docs/tools/virtiofsd.rst b/docs/tools/virtiofsd.rst
> > > > index a3a120da2f..5cb64612ed 100644
> > > > --- a/docs/tools/virtiofsd.rst
> > > > +++ b/docs/tools/virtiofsd.rst
> > > > @@ -163,6 +163,56 @@ in which case a 'server' rule will always match on all names from
> > > >  the server.
> > > >  
> > > >  
> > > > +xattr-mapping Examples
> > > > +----------------------
> > > > +
> > > > +1) Prefix all attributes with 'user.virtiofs.'
> > > > +
> > > > +::
> > > > +
> > > > +-o xattrmap=":prefix:all::user.virtiofs.::bad:all:::"
> > > > +
> > > 
> > > Staring at rule.
> > > 
> > > ":bad:all:::"
> > > 
> > > and trying to map this to ":type:scope:key:prepend:"
> > > 
> > > type==bad
> > > scope==all
> > > key==""
> > > prepend==""
> > 
> > Correct.
> > 
> > > > +
> > > > +This uses two rules, using : as the field separator;
> > > > +the first rule prefixes and strips 'user.virtiofs.',
> > > > +the second rule hides any non-prefixed attributes that
> > > > +the host set.
> > > 
> > > What is non-prefixed xattr in this context. If client sends
> > > "security.selinux", is prefixed or non-prefixed.
> > 
> > Note that anything originating at the client (i.e. starting with "")
> > will get caught by the first rule and be prefixed with 'user.virtiofs.'
> 
> So first rule is.
> 
> :prefix:all::user.virtiofs.:
> 
> scope="all"
> key==""
> prepend="user.virtiofs."
> 
> Given scope, rule applies to both client and server. Given key is "",
> any income setxattr() request will be prefixed with "user.virtiofs".
> And how does this rule work for server (listxattr()). It will match
> prepend "user.virtiofs" and if that matches it will be stripped?

Right; 'prefix' strips for server, adds for client (as long as you have
'all', you can select either one on it's own with server/client rather
than all).

> 
> 
> > This second rule will only be triggered by an xattr name coming
> > from the server (i.e a listxattr) for a name that doesn't begin
> > with user.virtiofs. (i.e. didn't match the 1st rule for a server xattr).
> > This makes sure that only guest created xattr's (that were set and
> > had a prefix added) are then visible to the guest.
> 
> Ok, so if an xattr does not match first rule, then second rule says
> key="", prepend="" and that will match all xattrs. So anything which
> is not caught by first rule, will be caught by second rule and
> either rejected or filtered out.

Right.

> > 
> > > Documentation in first patch said.
> > > 
> > > +- 'bad' - If a client tries to use this name it's
> > > +  denied using EPERM; when the server passes an attribute
> > > +  name matching it's hidden.
> > > 
> > > I am not sure which xattr name will be blocked if key=="" and prepend==""
> > 
> > All of them; they're still matched at the beginning; as the first
> > patch says 'It maybe empty in which case a 'client' rule will always
> > match on client names'
> 
> Ok.
> 
> > 
> > > > +2) Prefix 'trusted.' attributes, allow others through
> > > > +
> > > > +::
> > > > +
> > > > +   "/prefix/all/trusted./user.virtiofs./
> > > > +    /bad/server//trusted./
> > > > +    /bad/client/user.virtiofs.//
> > > > +    /ok/all///"
> > > > +
> > > > +
> > > > +Here there are four rules, using / as the field
> > > > +separator, and also demonstrating that new lines can
> > > > +be included between rules.
> > > > +The first rule is the prefixing of 'trusted.' and
> > > > +stripping of 'user.virtiofs.'.
> > > 
> > > So this is bidrectional rule, right. For setxattr(), "trusted."
> > > will be replaced with "user.virtiofs" and for listxattr(),
> > > server will replace user.virtiofs with trusted. ?
> > 
> > prefixed not replaced; so it'll turn "trusted." into
> > "user.virtiofs.trusted." and strip it back off for listxattr.
> 
> Ok. Got it. I am wondering how will I specify these rules so that
> they work in nested configuration. Say I have L0 host, L1 guest and
> L2 guest. Say virtiofsd0 is running on L0 and virtiofsd1 is running
> on L1. 
> 
> I am wondering how will I specify the rules on virtiofsd0 and virtiofsd1
> so that it works. Will it be same or rules are level dependent.

I'm hoping it'll be the same, see below.

> > 
> > > > +The second rule hides unprefixed 'trusted.' attributes
> > > > +on the host.
> > > 
> > > If host has "trusted.*", we are not hiding it and as per first
> > > rule we are converting it to "user.virtiofs.trusted.*", right?
> > > So why this second rule is needed.
> > 
> > No, the first rule will only prefix strings provided by the guest
> > and strip strings provided by the server. This rule hides
> > existing server 'trusted.' xattrs - so if the guest sets
> > trusted.foo it's not confused by also seeing a server trusted.foo
> > 
> > > > +The third rule stops a guest from explicitly setting
> > > > +the 'user.viritofs.' path directly.
> > > > +Finally, the fourth rule lets all remaining attributes
> > > > +through.
> > > 
> > > So If I don't specify third rule, and client does
> > > setxattr(user.virtiofs.*), it will simply be a passthrough?
> > 
> > Right; and that's dangerous, because a non-privileged guest
> > process can set a user. xattr; so a non-priv guest process could
> > set user.virtiofs.trusted.foo and then it would get read back
> > and be used as trusted.foo that has an impact on priviliged processes.
> 
> Right. We don't want unpriviliged process to be able to setup
> user.virtiofs.trusted.*. But that's what precisely happen in
> a nested configuration.
> 
> In above example, L2 will set trusted.foo, virtiofsd1 wil convert it
> to user.virtiofs.trusted.foo and virtiofsd0 will reject it, breaking
> the nested virtiofs.

So to allow nesting you need to nest the user.virtiofs. as well, not
just the trusted. So either you do an all, or if you want to be more
selective then I think the following would work:

 1  /prefix/client/trusted./user.virtiofs./
 2  /prefix/client/user.virtiofs./user.virtiofs./
 3  /prefix/server//user.virtiofs./
 4  /bad/server//trusted./
 5  /ok/all///

1 causes any getattr/setattr to convert 'trusted.'
                                   to   'user.virtiofs.trusted.'
2 causes any getattr/setattr to convert 'user.virtiofs.'
                                   to   'user.virtiofs.user.virtiofs.'
3 causes any listattr to lose a layer of user.virtiofs.
4 blocks any trusted. from the layer beneath
5 lets anything else through

(I'm trying to convince myself if we need a
/bad/server//user.virtiofs.trusted.  to stop the previous level being
visible).

Dave



> Thanks
> Vivek
Vivek Goyal Oct. 21, 2020, 1:44 p.m. UTC | #8
On Tue, Oct 20, 2020 at 08:02:37PM +0100, Dr. David Alan Gilbert wrote:

[..]
> > > > > +2) Prefix 'trusted.' attributes, allow others through
> > > > > +
> > > > > +::
> > > > > +
> > > > > +   "/prefix/all/trusted./user.virtiofs./
> > > > > +    /bad/server//trusted./
> > > > > +    /bad/client/user.virtiofs.//
> > > > > +    /ok/all///"
> > > > > +
> > > > > +
> > > > > +Here there are four rules, using / as the field
> > > > > +separator, and also demonstrating that new lines can
> > > > > +be included between rules.
> > > > > +The first rule is the prefixing of 'trusted.' and
> > > > > +stripping of 'user.virtiofs.'.
> > > > 
> > > > So this is bidrectional rule, right. For setxattr(), "trusted."
> > > > will be replaced with "user.virtiofs" and for listxattr(),
> > > > server will replace user.virtiofs with trusted. ?
> > > 
> > > prefixed not replaced; so it'll turn "trusted." into
> > > "user.virtiofs.trusted." and strip it back off for listxattr.
> > 
> > Ok. Got it. I am wondering how will I specify these rules so that
> > they work in nested configuration. Say I have L0 host, L1 guest and
> > L2 guest. Say virtiofsd0 is running on L0 and virtiofsd1 is running
> > on L1. 
> > 
> > I am wondering how will I specify the rules on virtiofsd0 and virtiofsd1
> > so that it works. Will it be same or rules are level dependent.
> 
> I'm hoping it'll be the same, see below.
> 
> > > 
> > > > > +The second rule hides unprefixed 'trusted.' attributes
> > > > > +on the host.
> > > > 
> > > > If host has "trusted.*", we are not hiding it and as per first
> > > > rule we are converting it to "user.virtiofs.trusted.*", right?
> > > > So why this second rule is needed.
> > > 
> > > No, the first rule will only prefix strings provided by the guest
> > > and strip strings provided by the server. This rule hides
> > > existing server 'trusted.' xattrs - so if the guest sets
> > > trusted.foo it's not confused by also seeing a server trusted.foo
> > > 
> > > > > +The third rule stops a guest from explicitly setting
> > > > > +the 'user.viritofs.' path directly.
> > > > > +Finally, the fourth rule lets all remaining attributes
> > > > > +through.
> > > > 
> > > > So If I don't specify third rule, and client does
> > > > setxattr(user.virtiofs.*), it will simply be a passthrough?
> > > 
> > > Right; and that's dangerous, because a non-privileged guest
> > > process can set a user. xattr; so a non-priv guest process could
> > > set user.virtiofs.trusted.foo and then it would get read back
> > > and be used as trusted.foo that has an impact on priviliged processes.
> > 
> > Right. We don't want unpriviliged process to be able to setup
> > user.virtiofs.trusted.*. But that's what precisely happen in
> > a nested configuration.
> > 
> > In above example, L2 will set trusted.foo, virtiofsd1 wil convert it
> > to user.virtiofs.trusted.foo and virtiofsd0 will reject it, breaking
> > the nested virtiofs.
> 
> So to allow nesting you need to nest the user.virtiofs. as well, not
> just the trusted. So either you do an all, or if you want to be more
> selective then I think the following would work:
> 
>  1  /prefix/client/trusted./user.virtiofs./
>  2  /prefix/client/user.virtiofs./user.virtiofs./

Ok, so basically instead of blocking user.virtiofs.trusted. from client,
prefix it with "user.virtiofs." one more time. IOW, allow client to
set user.virtiofs.trusted. because it will get back user.virtiofs.trusted.
and not "trusted." which is ok. Now client user space can't fool client
kernel with setting arbitrary user.virtiofs.trusted xattrs.

And if client kernel sends, trusted., it will get back trusted.

Only thing which can happen is that client1 sets user.virtiofs.trusted.
and nested client2 will get it as trusted. So client1 user space can
fool nested client's kernel. But given client1 has launched nested
client2, we should be able to trust some user on client1 and make
sure other users can't see this shared dir and this probably is
not an issue.

>  3  /prefix/server//user.virtiofs./
>  4  /bad/server//trusted./
>  5  /ok/all///
> 
> 1 causes any getattr/setattr to convert 'trusted.'
>                                    to   'user.virtiofs.trusted.'
> 2 causes any getattr/setattr to convert 'user.virtiofs.'
>                                    to   'user.virtiofs.user.virtiofs.'
> 3 causes any listattr to lose a layer of user.virtiofs.
> 4 blocks any trusted. from the layer beneath
> 5 lets anything else through
> 
> (I'm trying to convince myself if we need a
> /bad/server//user.virtiofs.trusted.  to stop the previous level being
> visible).

user.virtiofs.trusted on server will be converted to trusted., right?
Can't block it otherwise L1 client breaks, isn't it?

Vivek
Dr. David Alan Gilbert Oct. 21, 2020, 5:13 p.m. UTC | #9
* Stefan Hajnoczi (stefanha@redhat.com) wrote:
> On Wed, Oct 14, 2020 at 07:02:05PM +0100, Dr. David Alan Gilbert (git) wrote:
> > From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> > 
> > Add an option to define mappings of xattr names so that
> > the client and server filesystems see different views.
> > This can be used to have different SELinux mappings as
> > seen by the guest, to run the virtiofsd with less privileges
> > (e.g. in a case where it can't set trusted/system/security
> > xattrs but you want the guest to be able to), or to isolate
> > multiple users of the same name; e.g. trusted attributes
> > used by stacking overlayfs.
> > 
> > A mapping engine is used wit 3 simple rules; the rules can
> 
> s/wit/with/

Done.

> > +``:type:scope:key:prepend:``
> > +
> > +**type** is one of:
> > +
> > +- 'prefix' - If 'key' matches the client then the 'prepend'
> > +  is added before the name is passed to the server.
> > +  For a server case, the prepend is tested and stripped
> > +  if matching.
> 
> It may be clearer to document rule types like this:
> 
>   - :prefix:client:key:prepend: - Add 'prepend' before the name if it
>                                   starts with 'key'.
> 
>   - :prefix:server::prepend: - Strip 'prepend' if the name starts with
>                                it.
> 
> The client vs server behavior is independent so it's clearer to list
> them as separate cases. In addition, using the full rule syntax shows
> which fields are valid arguments and which ones are ignored.
> 
> The 'all' scope can be documented later as "Combines both the 'client'
> and 'server' scope behavior".

OK, I've reworked this quite a bit, into a simpler part for each of the
type entries with examples of each below.

> > +
> > +- 'ok' - The attribute name is OK and passed through to
> > +  the server unchanged.
> 
> The documentation isn't explicit but I think the default behavior is to
> allow all xattr names?
> 
> What is the purpose of the 'ok' rule? I guess it's to define an
> exception to a later 'prefix' or 'bad' rule. It would be nice to make
> this clear.
> 
> The documentation only mentions :client: behavior, leaving :server:
> undefined. Please indicate whether this rule has an effect in server
> scope.

What I have now is:
+**scope** is:
+
+- 'client' - match 'key' against a xattr name from the client for
+             setxattr/getxattr/removexattr
+- 'server' - match 'prepend' against a xattr name from the server
+             for listxattr
+- 'all' - can be used to make a single rule where both the server
+          and client matches are triggered.
+
+**type** is one of:
+
+- 'prefix' - is designed to prepend and strip a prefix;  the modified
+  attributes then being passed on to the client/server.
+
+- 'ok' - Causes the rule set to be terminated when a match is found
+  while allowing matching xattr's through unchanged.
+  It is intended both as a way of explicitly terminating
+  the list of rules, and to allow some xattr's to skip following rules.
+
+- 'bad' - If a client tries to use a name matching 'key' it's
+  denied using EPERM; when the server passes an attribute
+  name matching 'prepend' it's hidden.  In many ways it's use is very like
+  'ok' as either an explict terminator or for special handling of certain
+  patterns.
+
+**key** is a string tested as a prefix on an attribute name originating
+on the client.  It maybe empty in which case a 'client' rule
+will always match on client names.
+
+**prepend** is a string tested as a prefix on an attribute name originating
+on the server, and used as a new prefix.  It may be empty
+in which case a 'server' rule will always match on all names from
+the server.
+
+e.g.:
+
+  ``:prefix:client:trusted.:user.virtiofs.:``
+
+  will match 'trusted.' attributes in client calls and prefix them before
+  passing them to the server.
+
+  ``:prefix:server::user.virtiofs.:``
+
+  will strip 'user.virtiofs.' from all server replies.
+
+  ``:prefix:all:trusted.:user.virtiofs.:``
+
+  combines the previous two cases into a single rule.
+
+  ``:ok:client:user.::``
+
+  will allow get/set xattr for 'user.' xattr's and ignore
+  following rules.
+
+  ``:ok:server::security.:``
+
+  will pass 'securty.' xattr's in listxattr from the server
+  and ignore following rules.
+
+  ``:ok:all:::``
+
+  will terminate the rule search passing any remaining attributes
+  in both directions.
+
+  ``:bad:server::security.:``
+
+  would hide 'security.' xattr's in listxattr from the server.

so I'm hoping that addresses both the prefix and OK sections
at least.

> > +
> > +- 'bad' - If a client tries to use this name it's
> > +  denied using EPERM; when the server passes an attribute
> > +  name matching it's hidden.
> > +
> > +**scope** is:
> > +
> > +- 'client' - match 'key' against a xattr name from the client for
> > +             setxattr/getxattr/removexattr
> > +- 'server' - match 'prepend' against a xattr name from the server
> > +             for listxattr
> > +- 'all' - can be used to match both cases.
> > +
> > +**key** is a string tested as a prefix on an attribute name originating
> > +on the client.  It maybe empty in which case a 'client' rule
> > +will always match on client names.
> 
> Is there a way to match a full string instead of a prefix (regexp
> ^<pattern>$ instead of ^<pattern>)?

No there isn't; can you think of a way of representing that in the
syntax without making it much more complex?

> > @@ -2010,6 +2020,169 @@ static void lo_flock(fuse_req_t req, fuse_ino_t ino, struct fuse_file_info *fi,
> >      fuse_reply_err(req, res == -1 ? errno : 0);
> >  }
> >  
> > +/*
> > + * Exit; process attribute unmodified if matched.
> > + * An empty key applies to all.
> > + */
> > +#define XATTR_MAP_FLAG_END_OK  (1 <<  0)
> > +/*
> > + * The attribute is unwanted;
> > + * EPERM on write hidden on read.
> 
> Making this sentence easier to parse:
> 
> s/write hidden/write, hidden/

Done.

> 
> > + */
> > +#define XATTR_MAP_FLAG_END_BAD (1 <<  1)
> > +/*
> > + * For attr that start with 'key' prepend 'prepend'
> > + * 'key' maybe empty to prepend for all attrs
> 
> s/maybe/may be/

Hmm OK.

> > +    /* Add a terminator to error in cases the user hasn't specified */
> > +    tmp_entry.flags = XATTR_MAP_FLAG_ALL | XATTR_MAP_FLAG_END_BAD |
> > +                      XATTR_MAP_FLAG_LAST;
> 
> The comment is slightly misleading. This entry must be added in all
> cases since it terminates the lo->xattr_map_list with the
> XATTR_MAP_FLAG_LAST flag. If we don't add this entry then
> free_xattrmap() will iterate beyond the end of lo->xattr_map_list.
> 
> Another approach is to set XATTR_MAP_FLAG_LAST in add_xattrmap_entry()
> (and clear it on the previous last entry). That way adding the 'bad'
> catch-all truly is optional and just for cases where the user hasn't
> defined a catch-all rule themselves.

I've changed the comment.

> > +    tmp_entry.key = g_strdup("");
> > +    tmp_entry.prepend = g_strdup("");
> > +    lo->xattr_map_list = add_xattrmap_entry(lo->xattr_map_list, &nentries,
> > +                                            &tmp_entry);
> > +
> > +    return res;
> > +}
> > +
> >  static void lo_getxattr(fuse_req_t req, fuse_ino_t ino, const char *name,
> >                          size_t size)
> >  {
> > @@ -2806,6 +2979,8 @@ static void fuse_lo_data_cleanup(struct lo_data *lo)
> >          close(lo->root.fd);
> >      }
> >  
> > +    free(lo->xattrmap);
> > +    free_xattrmap(lo->xattr_map_list);
> >      free(lo->source);
> >  }
> >  
> > @@ -2906,6 +3081,11 @@ int main(int argc, char *argv[])
> >      } else {
> >          lo.source = strdup("/");
> >      }
> > +
> > +    if (lo.xattrmap) {
> > +        lo.xattr_map_list = parse_xattrmap(&lo);
> > +    }
> 
> The function always returns NULL. Has this been tested?

Hmm; I moved that xattr_map_list late and only retested with the
'map' shortcut which still returned it. Fixed.

Dave
Dr. David Alan Gilbert Oct. 21, 2020, 5:39 p.m. UTC | #10
* Vivek Goyal (vgoyal@redhat.com) wrote:
> On Tue, Oct 20, 2020 at 08:02:37PM +0100, Dr. David Alan Gilbert wrote:

> 

> [..]

> > > > > > +2) Prefix 'trusted.' attributes, allow others through

> > > > > > +

> > > > > > +::

> > > > > > +

> > > > > > +   "/prefix/all/trusted./user.virtiofs./

> > > > > > +    /bad/server//trusted./

> > > > > > +    /bad/client/user.virtiofs.//

> > > > > > +    /ok/all///"

> > > > > > +

> > > > > > +

> > > > > > +Here there are four rules, using / as the field

> > > > > > +separator, and also demonstrating that new lines can

> > > > > > +be included between rules.

> > > > > > +The first rule is the prefixing of 'trusted.' and

> > > > > > +stripping of 'user.virtiofs.'.

> > > > > 

> > > > > So this is bidrectional rule, right. For setxattr(), "trusted."

> > > > > will be replaced with "user.virtiofs" and for listxattr(),

> > > > > server will replace user.virtiofs with trusted. ?

> > > > 

> > > > prefixed not replaced; so it'll turn "trusted." into

> > > > "user.virtiofs.trusted." and strip it back off for listxattr.

> > > 

> > > Ok. Got it. I am wondering how will I specify these rules so that

> > > they work in nested configuration. Say I have L0 host, L1 guest and

> > > L2 guest. Say virtiofsd0 is running on L0 and virtiofsd1 is running

> > > on L1. 

> > > 

> > > I am wondering how will I specify the rules on virtiofsd0 and virtiofsd1

> > > so that it works. Will it be same or rules are level dependent.

> > 

> > I'm hoping it'll be the same, see below.

> > 

> > > > 

> > > > > > +The second rule hides unprefixed 'trusted.' attributes

> > > > > > +on the host.

> > > > > 

> > > > > If host has "trusted.*", we are not hiding it and as per first

> > > > > rule we are converting it to "user.virtiofs.trusted.*", right?

> > > > > So why this second rule is needed.

> > > > 

> > > > No, the first rule will only prefix strings provided by the guest

> > > > and strip strings provided by the server. This rule hides

> > > > existing server 'trusted.' xattrs - so if the guest sets

> > > > trusted.foo it's not confused by also seeing a server trusted.foo

> > > > 

> > > > > > +The third rule stops a guest from explicitly setting

> > > > > > +the 'user.viritofs.' path directly.

> > > > > > +Finally, the fourth rule lets all remaining attributes

> > > > > > +through.

> > > > > 

> > > > > So If I don't specify third rule, and client does

> > > > > setxattr(user.virtiofs.*), it will simply be a passthrough?

> > > > 

> > > > Right; and that's dangerous, because a non-privileged guest

> > > > process can set a user. xattr; so a non-priv guest process could

> > > > set user.virtiofs.trusted.foo and then it would get read back

> > > > and be used as trusted.foo that has an impact on priviliged processes.

> > > 

> > > Right. We don't want unpriviliged process to be able to setup

> > > user.virtiofs.trusted.*. But that's what precisely happen in

> > > a nested configuration.

> > > 

> > > In above example, L2 will set trusted.foo, virtiofsd1 wil convert it

> > > to user.virtiofs.trusted.foo and virtiofsd0 will reject it, breaking

> > > the nested virtiofs.

> > 

> > So to allow nesting you need to nest the user.virtiofs. as well, not

> > just the trusted. So either you do an all, or if you want to be more

> > selective then I think the following would work:

> > 

> >  1  /prefix/client/trusted./user.virtiofs./

> >  2  /prefix/client/user.virtiofs./user.virtiofs./

> 

> Ok, so basically instead of blocking user.virtiofs.trusted. from client,

> prefix it with "user.virtiofs." one more time. IOW, allow client to

> set user.virtiofs.trusted. because it will get back user.virtiofs.trusted.

> and not "trusted." which is ok. Now client user space can't fool client

> kernel with setting arbitrary user.virtiofs.trusted xattrs.


Right.

> And if client kernel sends, trusted., it will get back trusted.


Right.

> Only thing which can happen is that client1 sets user.virtiofs.trusted.

> and nested client2 will get it as trusted. So client1 user space can

> fool nested client's kernel. But given client1 has launched nested

> client2, we should be able to trust some user on client1 and make

> sure other users can't see this shared dir and this probably is

> not an issue.


Yes, that does depend a bit on how you're intending to share your
filesystems etc

> >  3  /prefix/server//user.virtiofs./

> >  4  /bad/server//trusted./

> >  5  /ok/all///

> > 

> > 1 causes any getattr/setattr to convert 'trusted.'

> >                                    to   'user.virtiofs.trusted.'

> > 2 causes any getattr/setattr to convert 'user.virtiofs.'

> >                                    to   'user.virtiofs.user.virtiofs.'

> > 3 causes any listattr to lose a layer of user.virtiofs.

> > 4 blocks any trusted. from the layer beneath

> > 5 lets anything else through

> > 

> > (I'm trying to convince myself if we need a

> > /bad/server//user.virtiofs.trusted.  to stop the previous level being

> > visible).

> 

> user.virtiofs.trusted on server will be converted to trusted., right?

> Can't block it otherwise L1 client breaks, isn't it?


True.

Dave

> Vivek

-- 
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
Vivek Goyal Oct. 22, 2020, 1:42 p.m. UTC | #11
On Wed, Oct 14, 2020 at 07:02:09PM +0100, Dr. David Alan Gilbert (git) wrote:
> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> 
> The mapping rule system implemented in the last few patches is
> extremely flexible, but not easy to use.  Add a simple
> 'map' type as a sprinkling of sugar to make it easy.
> 
> e.g.
> 
>   -o xattrmap=":map::user.virtiofs.:"
> 
> would be sufficient to prefix all xattr's
> or
> 
>   -o xattrmap=":map:trusted.:user.virtiofs.:"
> 
> would just prefix 'trusted.' xattr's and leave
> everything else alone.

Will it block "user.virtiofs.trusted." from client? As we discussed
that either we need to block it or we need to prefix it with another
user.virtiofs. I mean this rule alone is problematic and needs
to be coupled with more rules.

I am assuming one can specify multiple xattrmap on single line. So
one can also say.

-o xattrmap=":map:trusted.:user.virtiofs.:" -o xattrmap=":map:user.virtiofs.:user.virtiofs."

Thanks
Vivek

> 
> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> ---
>  docs/tools/virtiofsd.rst         |  18 ++++++
>  tools/virtiofsd/passthrough_ll.c | 108 ++++++++++++++++++++++++++++++-
>  2 files changed, 125 insertions(+), 1 deletion(-)
> 
> diff --git a/docs/tools/virtiofsd.rst b/docs/tools/virtiofsd.rst
> index 5cb64612ed..e388ef253e 100644
> --- a/docs/tools/virtiofsd.rst
> +++ b/docs/tools/virtiofsd.rst
> @@ -127,6 +127,7 @@ Each rule consists of a number of fields separated with a separator that is the
>  first non-white space character in the rule.  This separator must then be used
>  for the whole rule.
>  White space may be added before and after each rule.
> +
>  Using ':' as the separator a rule is of the form:
>  
>  ``:type:scope:key:prepend:``
> @@ -162,6 +163,13 @@ on the server, and used as a new prefix.  It may be empty
>  in which case a 'server' rule will always match on all names from
>  the server.
>  
> +A simpler 'map' type provides a shorter syntax for the common case:
> +
> +``:map:key:prepend:``
> +
> +The 'map' type adds a number of separate rules to add **prepend** as a prefix
> +to the matched **key** (or all attributes if **key** is empty).
> +There may be at most one 'map' rule and it must be the last rule in the set.
>  
>  xattr-mapping Examples
>  ----------------------
> @@ -178,6 +186,11 @@ the first rule prefixes and strips 'user.virtiofs.',
>  the second rule hides any non-prefixed attributes that
>  the host set.
>  
> +This is equivalent to the 'map' rule:
> +
> +::
> +-o xattrmap=":map::user.virtiofs.:"
> +
>  2) Prefix 'trusted.' attributes, allow others through
>  
>  ::
> @@ -200,6 +213,11 @@ the 'user.viritofs.' path directly.
>  Finally, the fourth rule lets all remaining attributes
>  through.
>  
> +This is equivalent to the 'map' rule:
> +
> +::
> +-o xattrmap="/map/trusted./user.virtiofs./"
> +
>  3) Hide 'security.' attributes, and allow everything else
>  
>  ::
> diff --git a/tools/virtiofsd/passthrough_ll.c b/tools/virtiofsd/passthrough_ll.c
> index 8406a2ae86..a1b3364ba3 100644
> --- a/tools/virtiofsd/passthrough_ll.c
> +++ b/tools/virtiofsd/passthrough_ll.c
> @@ -2074,6 +2074,106 @@ static void free_xattrmap(XattrMapEntry *map)
>      g_free(map);
>  }
>  
> +/*
> + * Handle the 'map' type, which is sugar for a set of commands
> + * for the common case of prefixing a subset or everything,
> + * and allowing anything not prefixed through.
> + * It must be the last entry in the stream, although there
> + * can be other entries before it.
> + * The form is:
> + *    :map:key:prefix:
> + *
> + * key maybe empty in which case all entries are prefixed.
> + */
> +static XattrMapEntry *parse_xattrmap_map(const char *rule,
> +                                         XattrMapEntry *map,
> +                                         size_t *nentries)
> +{
> +    char sep = *rule++;
> +    const char *tmp;
> +    char *key;
> +    char *prefix;
> +    XattrMapEntry tmp_entry;
> +
> +    /* At start of 'key' field */
> +    tmp = strchr(rule, sep);
> +    if (!tmp) {
> +        fuse_log(FUSE_LOG_ERR,
> +                 "%s: Missing '%c' at end of key field in map rule\n",
> +                 __func__, sep);
> +        exit(1);
> +    }
> +
> +    key = g_strndup(rule, tmp - rule);
> +    rule = tmp + 1;
> +
> +    /* At start of prefix field */
> +    tmp = strchr(rule, sep);
> +    if (!tmp) {
> +        fuse_log(FUSE_LOG_ERR,
> +                 "%s: Missing '%c' at end of prefix field in map rule\n",
> +                 __func__, sep);
> +        exit(1);
> +    }
> +
> +    prefix = g_strndup(rule, tmp - rule);
> +    rule = tmp + 1;
> +
> +    /*
> +     * This should be the end of the string, we don't allow
> +     * any more commands after 'map'.
> +     */
> +    if (*rule) {
> +        fuse_log(FUSE_LOG_ERR,
> +                 "%s: Expecting end of command after map, found '%c'\n",
> +                 __func__, *rule);
> +        exit(1);
> +    }
> +
> +    /* 1st: Prefix matches/everything */
> +    tmp_entry.flags = XATTR_MAP_FLAG_PREFIX | XATTR_MAP_FLAG_ALL;
> +    tmp_entry.key = g_strdup(key);
> +    tmp_entry.prepend = g_strdup(prefix);
> +    map = add_xattrmap_entry(map, nentries, &tmp_entry);
> +
> +    if (!*key) {
> +        /* Prefix all case */
> +
> +        /* 2nd: Hide any non-prefixed entries on the host */
> +        tmp_entry.flags = XATTR_MAP_FLAG_END_BAD | XATTR_MAP_FLAG_ALL |
> +                          XATTR_MAP_FLAG_LAST;
> +        tmp_entry.key = g_strdup("");
> +        tmp_entry.prepend = g_strdup("");
> +        map = add_xattrmap_entry(map, nentries, &tmp_entry);
> +    } else {
> +        /* Prefix matching case */
> +
> +        /* 2nd: Hide non-prefixed but matching entries on the host */
> +        tmp_entry.flags = XATTR_MAP_FLAG_END_BAD | XATTR_MAP_FLAG_SERVER;
> +        tmp_entry.key = g_strdup(""); /* Not used */
> +        tmp_entry.prepend = g_strdup(key);
> +        map = add_xattrmap_entry(map, nentries, &tmp_entry);
> +
> +        /* 3rd: Stop the client accessing prefixed attributes directly */
> +        tmp_entry.flags = XATTR_MAP_FLAG_END_BAD | XATTR_MAP_FLAG_CLIENT;
> +        tmp_entry.key = g_strdup(prefix);
> +        tmp_entry.prepend = g_strdup(""); /* Not used */
> +        map = add_xattrmap_entry(map, nentries, &tmp_entry);
> +
> +        /* 4th: Everything else is OK */
> +        tmp_entry.flags = XATTR_MAP_FLAG_END_OK | XATTR_MAP_FLAG_ALL |
> +                          XATTR_MAP_FLAG_LAST;
> +        tmp_entry.key = g_strdup("");
> +        tmp_entry.prepend = g_strdup("");
> +        map = add_xattrmap_entry(map, nentries, &tmp_entry);
> +    }
> +
> +    g_free(key);
> +    g_free(prefix);
> +
> +    return map;
> +}
> +
>  static XattrMapEntry *parse_xattrmap(struct lo_data *lo)
>  {
>      XattrMapEntry *res = NULL;
> @@ -2102,10 +2202,16 @@ static XattrMapEntry *parse_xattrmap(struct lo_data *lo)
>              tmp_entry.flags |= XATTR_MAP_FLAG_END_OK;
>          } else if (strstart(map, "bad", &map)) {
>              tmp_entry.flags |= XATTR_MAP_FLAG_END_BAD;
> +        } else if (strstart(map, "map", &map)) {
> +            /*
> +             * map is sugar that adds a number of rules, and must be
> +             * the last entry.
> +             */
> +            return parse_xattrmap_map(map, res, &nentries);
>          } else {
>              fuse_log(FUSE_LOG_ERR,
>                       "%s: Unexpected type;"
> -                     "Expecting 'prefix', 'ok', or 'bad' in rule %zu\n",
> +                     "Expecting 'prefix', 'ok', 'bad' or 'map' in rule %zu\n",
>                       __func__, nentries);
>              exit(1);
>          }
> -- 
> 2.28.0
>
Vivek Goyal Oct. 22, 2020, 2:52 p.m. UTC | #12
On Wed, Oct 14, 2020 at 07:02:05PM +0100, Dr. David Alan Gilbert (git) wrote:

[..]
> +/*
> + * Exit; process attribute unmodified if matched.
> + * An empty key applies to all.
> + */
> +#define XATTR_MAP_FLAG_END_OK  (1 <<  0)
> +/*
> + * The attribute is unwanted;
> + * EPERM on write hidden on read.
> + */
> +#define XATTR_MAP_FLAG_END_BAD (1 <<  1)
> +/*
> + * For attr that start with 'key' prepend 'prepend'
> + * 'key' maybe empty to prepend for all attrs
> + * key is defined from set/remove point of view.
> + * Automatically reversed on read
> + */
> +#define XATTR_MAP_FLAG_PREFIX  (1 <<  2)
> +/* Apply rule to get/set/remove */
> +#define XATTR_MAP_FLAG_CLIENT  (1 << 16)
> +/* Apply rule to list */
> +#define XATTR_MAP_FLAG_SERVER  (1 << 17)
> +/* Apply rule to all */
> +#define XATTR_MAP_FLAG_ALL   (XATTR_MAP_FLAG_SERVER | XATTR_MAP_FLAG_CLIENT)
> +
> +/* Last rule in the XATTR_MAP */
> +#define XATTR_MAP_FLAG_LAST    (1 << 30)

I see that you are using bit positions for flags. Not clear why you
used bit 0,1,2 and then jumped to 16,17 and then to 30. May be you
are doing some sort of reservation of bits. Will be nice to explain
that a bit so that next person modifying it can use bits from
correct pool.

> +
> +static XattrMapEntry *add_xattrmap_entry(XattrMapEntry *orig_map,
> +                                         size_t *nentries,
> +                                         const XattrMapEntry *new_entry)
> +{
> +    XattrMapEntry *res = g_realloc_n(orig_map, ++*nentries,
> +                                     sizeof(XattrMapEntry));
> +    res[*nentries - 1] = *new_entry;
> +
> +    return res;
> +}
> +
> +static void free_xattrmap(XattrMapEntry *map)
> +{
> +    XattrMapEntry *curr = map;
> +
> +    if (!map) {
> +        return;
> +    };

; after } is not needed.

> +
> +    do {
> +        g_free(curr->key);
> +        g_free(curr->prepend);
> +    } while (!(curr++->flags & XATTR_MAP_FLAG_LAST));
> +
> +    g_free(map);
> +}
> +
> +static XattrMapEntry *parse_xattrmap(struct lo_data *lo)
> +{
> +    XattrMapEntry *res = NULL;
> +    XattrMapEntry tmp_entry;
> +    size_t nentries = 0;

If you are calculating number of entries (nentries), may be this could
be stored in lo_data so that can be later used to free entries or loop
through rules etc.

> +    const char *map = lo->xattrmap;
> +    const char *tmp;
> +
> +    while (*map) {
> +        char sep;
> +
> +        if (isspace(*map)) {
> +            map++;
> +            continue;
> +        }
> +        /* The separator is the first non-space of the rule */
> +        sep = *map++;
> +        if (!sep) {
> +            break;
> +        }

When can sep be NULL? In that case while loop will not even continue.

> +
> +        /* Start of 'type' */
> +        if (strstart(map, "prefix", &map)) {
> +            tmp_entry.flags |= XATTR_MAP_FLAG_PREFIX;
> +        } else if (strstart(map, "ok", &map)) {
> +            tmp_entry.flags |= XATTR_MAP_FLAG_END_OK;
> +        } else if (strstart(map, "bad", &map)) {
> +            tmp_entry.flags |= XATTR_MAP_FLAG_END_BAD;
> +        } else {
> +            fuse_log(FUSE_LOG_ERR,
> +                     "%s: Unexpected type;"
> +                     "Expecting 'prefix', 'ok', or 'bad' in rule %zu\n",
> +                     __func__, nentries);
> +            exit(1);
> +        }
> +
> +        if (*map++ != sep) {
> +            fuse_log(FUSE_LOG_ERR,
> +                     "%s: Missing '%c' at end of type field of rule %zu\n",
> +                     __func__, sep, nentries);
> +            exit(1);
> +        }
> +
> +        /* Start of 'scope' */
> +        if (strstart(map, "client", &map)) {
> +            tmp_entry.flags |= XATTR_MAP_FLAG_CLIENT;
> +        } else if (strstart(map, "server", &map)) {
> +            tmp_entry.flags |= XATTR_MAP_FLAG_SERVER;
> +        } else if (strstart(map, "all", &map)) {
> +            tmp_entry.flags |= XATTR_MAP_FLAG_ALL;
> +        } else {
> +            fuse_log(FUSE_LOG_ERR,
> +                     "%s: Unexpected scope;"
> +                     " Expecting 'client', 'server', or 'all', in rule %zu\n",
> +                     __func__, nentries);
> +            exit(1);
> +        }
> +
> +        if (*map++ != sep) {
> +            fuse_log(FUSE_LOG_ERR,
> +                     "%s: Expecting '%c' found '%c'"
> +                     " after scope in rule %zu\n",
> +                     __func__, sep, *map, nentries);
> +            exit(1);
> +        }
> +
> +        /* At start of 'key' field */
> +        tmp = strchr(map, sep);
> +        if (!tmp) {
> +            fuse_log(FUSE_LOG_ERR,
> +                     "%s: Missing '%c' at end of key field of rule %zu",
> +                     __func__, sep, nentries);
> +            exit(1);
> +        }
> +        tmp_entry.key = g_strndup(map, tmp - map);
> +        map = tmp + 1;
> +
> +        /* At start of 'prepend' field */
> +        tmp = strchr(map, sep);
> +        if (!tmp) {
> +            fuse_log(FUSE_LOG_ERR,
> +                     "%s: Missing '%c' at end of prepend field of rule %zu",
> +                     __func__, sep, nentries);
> +            exit(1);
> +        }
> +        tmp_entry.prepend = g_strndup(map, tmp - map);
> +        map = tmp + 1;
> +
> +        lo->xattr_map_list = add_xattrmap_entry(lo->xattr_map_list, &nentries,
> +                                                &tmp_entry);
> +        /* End of rule - go around again for another rule */
> +    }
> +
> +    if (!nentries) {
> +        fuse_log(FUSE_LOG_ERR, "Empty xattr map\n");
> +        exit(1);
> +    }
> +
> +    /* Add a terminator to error in cases the user hasn't specified */
> +    tmp_entry.flags = XATTR_MAP_FLAG_ALL | XATTR_MAP_FLAG_END_BAD |
> +                      XATTR_MAP_FLAG_LAST;
> +    tmp_entry.key = g_strdup("");
> +    tmp_entry.prepend = g_strdup("");
> +    lo->xattr_map_list = add_xattrmap_entry(lo->xattr_map_list, &nentries,
> +                                            &tmp_entry);

Not sure why this default rule is needed when user has not specified one.

This seems to be equivalent of ":bad:all:::". Will this not block all
the xattrs which have not been caught by previous rules. And user
probably did not want it. 

Thanks
Vivek
Dr. David Alan Gilbert Oct. 23, 2020, 1:05 p.m. UTC | #13
* Vivek Goyal (vgoyal@redhat.com) wrote:
> On Wed, Oct 14, 2020 at 07:02:09PM +0100, Dr. David Alan Gilbert (git) wrote:
> > From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> > 
> > The mapping rule system implemented in the last few patches is
> > extremely flexible, but not easy to use.  Add a simple
> > 'map' type as a sprinkling of sugar to make it easy.
> > 
> > e.g.
> > 
> >   -o xattrmap=":map::user.virtiofs.:"
> > 
> > would be sufficient to prefix all xattr's
> > or
> > 
> >   -o xattrmap=":map:trusted.:user.virtiofs.:"
> > 
> > would just prefix 'trusted.' xattr's and leave
> > everything else alone.
> 
> Will it block "user.virtiofs.trusted." from client? As we discussed
> that either we need to block it or we need to prefix it with another
> user.virtiofs. I mean this rule alone is problematic and needs
> to be coupled with more rules.

It blocked user.virtiofs. more generically:

+        /* 2nd: Hide non-prefixed but matching entries on the host */
+        tmp_entry.flags = XATTR_MAP_FLAG_END_BAD | XATTR_MAP_FLAG_SERVER;
+        tmp_entry.key = g_strdup(""); /* Not used */
+        tmp_entry.prepend = g_strdup(key);
+        map = add_xattrmap_entry(map, nentries, &tmp_entry);

i.e. hides 'trusted.'

+        /* 3rd: Stop the client accessing prefixed attributes directly */
+        tmp_entry.flags = XATTR_MAP_FLAG_END_BAD | XATTR_MAP_FLAG_CLIENT;
+        tmp_entry.key = g_strdup(prefix);
+        tmp_entry.prepend = g_strdup(""); /* Not used */
+        map = add_xattrmap_entry(map, nentries, &tmp_entry);

i.e. blocks client from 'user.virtiofs.'

Note, that is a problem for nesting, since having that at L1 means it
stops L2 from settings user.virtiofs.trusted.something (which it wants
to become user.virtiofs.user.virtiofs.trusted.something).

Using :map::user.virtiofs.:  works nicely for nesting (I've just tried
it), since that nests everything.

> I am assuming one can specify multiple xattrmap on single line. So
> one can also say.
> 
> -o xattrmap=":map:trusted.:user.virtiofs.:" -o xattrmap=":map:user.virtiofs.:user.virtiofs."

No, the existing core option parser we're using overrides on subsequent
entries; so that would ignore the first -o xattrmap=

Dave

> 
> Thanks
> Vivek
> 
> > 
> > Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> > ---
> >  docs/tools/virtiofsd.rst         |  18 ++++++
> >  tools/virtiofsd/passthrough_ll.c | 108 ++++++++++++++++++++++++++++++-
> >  2 files changed, 125 insertions(+), 1 deletion(-)
> > 
> > diff --git a/docs/tools/virtiofsd.rst b/docs/tools/virtiofsd.rst
> > index 5cb64612ed..e388ef253e 100644
> > --- a/docs/tools/virtiofsd.rst
> > +++ b/docs/tools/virtiofsd.rst
> > @@ -127,6 +127,7 @@ Each rule consists of a number of fields separated with a separator that is the
> >  first non-white space character in the rule.  This separator must then be used
> >  for the whole rule.
> >  White space may be added before and after each rule.
> > +
> >  Using ':' as the separator a rule is of the form:
> >  
> >  ``:type:scope:key:prepend:``
> > @@ -162,6 +163,13 @@ on the server, and used as a new prefix.  It may be empty
> >  in which case a 'server' rule will always match on all names from
> >  the server.
> >  
> > +A simpler 'map' type provides a shorter syntax for the common case:
> > +
> > +``:map:key:prepend:``
> > +
> > +The 'map' type adds a number of separate rules to add **prepend** as a prefix
> > +to the matched **key** (or all attributes if **key** is empty).
> > +There may be at most one 'map' rule and it must be the last rule in the set.
> >  
> >  xattr-mapping Examples
> >  ----------------------
> > @@ -178,6 +186,11 @@ the first rule prefixes and strips 'user.virtiofs.',
> >  the second rule hides any non-prefixed attributes that
> >  the host set.
> >  
> > +This is equivalent to the 'map' rule:
> > +
> > +::
> > +-o xattrmap=":map::user.virtiofs.:"
> > +
> >  2) Prefix 'trusted.' attributes, allow others through
> >  
> >  ::
> > @@ -200,6 +213,11 @@ the 'user.viritofs.' path directly.
> >  Finally, the fourth rule lets all remaining attributes
> >  through.
> >  
> > +This is equivalent to the 'map' rule:
> > +
> > +::
> > +-o xattrmap="/map/trusted./user.virtiofs./"
> > +
> >  3) Hide 'security.' attributes, and allow everything else
> >  
> >  ::
> > diff --git a/tools/virtiofsd/passthrough_ll.c b/tools/virtiofsd/passthrough_ll.c
> > index 8406a2ae86..a1b3364ba3 100644
> > --- a/tools/virtiofsd/passthrough_ll.c
> > +++ b/tools/virtiofsd/passthrough_ll.c
> > @@ -2074,6 +2074,106 @@ static void free_xattrmap(XattrMapEntry *map)
> >      g_free(map);
> >  }
> >  
> > +/*
> > + * Handle the 'map' type, which is sugar for a set of commands
> > + * for the common case of prefixing a subset or everything,
> > + * and allowing anything not prefixed through.
> > + * It must be the last entry in the stream, although there
> > + * can be other entries before it.
> > + * The form is:
> > + *    :map:key:prefix:
> > + *
> > + * key maybe empty in which case all entries are prefixed.
> > + */
> > +static XattrMapEntry *parse_xattrmap_map(const char *rule,
> > +                                         XattrMapEntry *map,
> > +                                         size_t *nentries)
> > +{
> > +    char sep = *rule++;
> > +    const char *tmp;
> > +    char *key;
> > +    char *prefix;
> > +    XattrMapEntry tmp_entry;
> > +
> > +    /* At start of 'key' field */
> > +    tmp = strchr(rule, sep);
> > +    if (!tmp) {
> > +        fuse_log(FUSE_LOG_ERR,
> > +                 "%s: Missing '%c' at end of key field in map rule\n",
> > +                 __func__, sep);
> > +        exit(1);
> > +    }
> > +
> > +    key = g_strndup(rule, tmp - rule);
> > +    rule = tmp + 1;
> > +
> > +    /* At start of prefix field */
> > +    tmp = strchr(rule, sep);
> > +    if (!tmp) {
> > +        fuse_log(FUSE_LOG_ERR,
> > +                 "%s: Missing '%c' at end of prefix field in map rule\n",
> > +                 __func__, sep);
> > +        exit(1);
> > +    }
> > +
> > +    prefix = g_strndup(rule, tmp - rule);
> > +    rule = tmp + 1;
> > +
> > +    /*
> > +     * This should be the end of the string, we don't allow
> > +     * any more commands after 'map'.
> > +     */
> > +    if (*rule) {
> > +        fuse_log(FUSE_LOG_ERR,
> > +                 "%s: Expecting end of command after map, found '%c'\n",
> > +                 __func__, *rule);
> > +        exit(1);
> > +    }
> > +
> > +    /* 1st: Prefix matches/everything */
> > +    tmp_entry.flags = XATTR_MAP_FLAG_PREFIX | XATTR_MAP_FLAG_ALL;
> > +    tmp_entry.key = g_strdup(key);
> > +    tmp_entry.prepend = g_strdup(prefix);
> > +    map = add_xattrmap_entry(map, nentries, &tmp_entry);
> > +
> > +    if (!*key) {
> > +        /* Prefix all case */
> > +
> > +        /* 2nd: Hide any non-prefixed entries on the host */
> > +        tmp_entry.flags = XATTR_MAP_FLAG_END_BAD | XATTR_MAP_FLAG_ALL |
> > +                          XATTR_MAP_FLAG_LAST;
> > +        tmp_entry.key = g_strdup("");
> > +        tmp_entry.prepend = g_strdup("");
> > +        map = add_xattrmap_entry(map, nentries, &tmp_entry);
> > +    } else {
> > +        /* Prefix matching case */
> > +
> > +        /* 2nd: Hide non-prefixed but matching entries on the host */
> > +        tmp_entry.flags = XATTR_MAP_FLAG_END_BAD | XATTR_MAP_FLAG_SERVER;
> > +        tmp_entry.key = g_strdup(""); /* Not used */
> > +        tmp_entry.prepend = g_strdup(key);
> > +        map = add_xattrmap_entry(map, nentries, &tmp_entry);
> > +
> > +        /* 3rd: Stop the client accessing prefixed attributes directly */
> > +        tmp_entry.flags = XATTR_MAP_FLAG_END_BAD | XATTR_MAP_FLAG_CLIENT;
> > +        tmp_entry.key = g_strdup(prefix);
> > +        tmp_entry.prepend = g_strdup(""); /* Not used */
> > +        map = add_xattrmap_entry(map, nentries, &tmp_entry);
> > +
> > +        /* 4th: Everything else is OK */
> > +        tmp_entry.flags = XATTR_MAP_FLAG_END_OK | XATTR_MAP_FLAG_ALL |
> > +                          XATTR_MAP_FLAG_LAST;
> > +        tmp_entry.key = g_strdup("");
> > +        tmp_entry.prepend = g_strdup("");
> > +        map = add_xattrmap_entry(map, nentries, &tmp_entry);
> > +    }
> > +
> > +    g_free(key);
> > +    g_free(prefix);
> > +
> > +    return map;
> > +}
> > +
> >  static XattrMapEntry *parse_xattrmap(struct lo_data *lo)
> >  {
> >      XattrMapEntry *res = NULL;
> > @@ -2102,10 +2202,16 @@ static XattrMapEntry *parse_xattrmap(struct lo_data *lo)
> >              tmp_entry.flags |= XATTR_MAP_FLAG_END_OK;
> >          } else if (strstart(map, "bad", &map)) {
> >              tmp_entry.flags |= XATTR_MAP_FLAG_END_BAD;
> > +        } else if (strstart(map, "map", &map)) {
> > +            /*
> > +             * map is sugar that adds a number of rules, and must be
> > +             * the last entry.
> > +             */
> > +            return parse_xattrmap_map(map, res, &nentries);
> >          } else {
> >              fuse_log(FUSE_LOG_ERR,
> >                       "%s: Unexpected type;"
> > -                     "Expecting 'prefix', 'ok', or 'bad' in rule %zu\n",
> > +                     "Expecting 'prefix', 'ok', 'bad' or 'map' in rule %zu\n",
> >                       __func__, nentries);
> >              exit(1);
> >          }
> > -- 
> > 2.28.0
> >
Dr. David Alan Gilbert Oct. 23, 2020, 3:46 p.m. UTC | #14
* Vivek Goyal (vgoyal@redhat.com) wrote:
> On Wed, Oct 14, 2020 at 07:02:05PM +0100, Dr. David Alan Gilbert (git) wrote:
> 
> [..]
> > +/*
> > + * Exit; process attribute unmodified if matched.
> > + * An empty key applies to all.
> > + */
> > +#define XATTR_MAP_FLAG_END_OK  (1 <<  0)
> > +/*
> > + * The attribute is unwanted;
> > + * EPERM on write hidden on read.
> > + */
> > +#define XATTR_MAP_FLAG_END_BAD (1 <<  1)
> > +/*
> > + * For attr that start with 'key' prepend 'prepend'
> > + * 'key' maybe empty to prepend for all attrs
> > + * key is defined from set/remove point of view.
> > + * Automatically reversed on read
> > + */
> > +#define XATTR_MAP_FLAG_PREFIX  (1 <<  2)
> > +/* Apply rule to get/set/remove */
> > +#define XATTR_MAP_FLAG_CLIENT  (1 << 16)
> > +/* Apply rule to list */
> > +#define XATTR_MAP_FLAG_SERVER  (1 << 17)
> > +/* Apply rule to all */
> > +#define XATTR_MAP_FLAG_ALL   (XATTR_MAP_FLAG_SERVER | XATTR_MAP_FLAG_CLIENT)
> > +
> > +/* Last rule in the XATTR_MAP */
> > +#define XATTR_MAP_FLAG_LAST    (1 << 30)
> 
> I see that you are using bit positions for flags. Not clear why you
> used bit 0,1,2 and then jumped to 16,17 and then to 30. May be you
> are doing some sort of reservation of bits. Will be nice to explain
> that a bit so that next person modifying it can use bits from
> correct pool.

I've added a 'types' and a 'scopes' comment pair to hopefully make it
clear how I split it up.

> > +
> > +static XattrMapEntry *add_xattrmap_entry(XattrMapEntry *orig_map,
> > +                                         size_t *nentries,
> > +                                         const XattrMapEntry *new_entry)
> > +{
> > +    XattrMapEntry *res = g_realloc_n(orig_map, ++*nentries,
> > +                                     sizeof(XattrMapEntry));
> > +    res[*nentries - 1] = *new_entry;
> > +
> > +    return res;
> > +}
> > +
> > +static void free_xattrmap(XattrMapEntry *map)
> > +{
> > +    XattrMapEntry *curr = map;
> > +
> > +    if (!map) {
> > +        return;
> > +    };
> 
> ; after } is not needed.

Gone.

> > +
> > +    do {
> > +        g_free(curr->key);
> > +        g_free(curr->prepend);
> > +    } while (!(curr++->flags & XATTR_MAP_FLAG_LAST));
> > +
> > +    g_free(map);
> > +}
> > +
> > +static XattrMapEntry *parse_xattrmap(struct lo_data *lo)
> > +{
> > +    XattrMapEntry *res = NULL;
> > +    XattrMapEntry tmp_entry;
> > +    size_t nentries = 0;
> 
> If you are calculating number of entries (nentries), may be this could
> be stored in lo_data so that can be later used to free entries or loop
> through rules etc.

Done; and that removes the need for _LAST.

> > +    const char *map = lo->xattrmap;
> > +    const char *tmp;
> > +
> > +    while (*map) {
> > +        char sep;
> > +
> > +        if (isspace(*map)) {
> > +            map++;
> > +            continue;
> > +        }
> > +        /* The separator is the first non-space of the rule */
> > +        sep = *map++;
> > +        if (!sep) {
> > +            break;
> > +        }
> 
> When can sep be NULL? In that case while loop will not even continue.

The end of the rule list.

> > +
> > +        /* Start of 'type' */
> > +        if (strstart(map, "prefix", &map)) {
> > +            tmp_entry.flags |= XATTR_MAP_FLAG_PREFIX;
> > +        } else if (strstart(map, "ok", &map)) {
> > +            tmp_entry.flags |= XATTR_MAP_FLAG_END_OK;
> > +        } else if (strstart(map, "bad", &map)) {
> > +            tmp_entry.flags |= XATTR_MAP_FLAG_END_BAD;
> > +        } else {
> > +            fuse_log(FUSE_LOG_ERR,
> > +                     "%s: Unexpected type;"
> > +                     "Expecting 'prefix', 'ok', or 'bad' in rule %zu\n",
> > +                     __func__, nentries);
> > +            exit(1);
> > +        }
> > +
> > +        if (*map++ != sep) {
> > +            fuse_log(FUSE_LOG_ERR,
> > +                     "%s: Missing '%c' at end of type field of rule %zu\n",
> > +                     __func__, sep, nentries);
> > +            exit(1);
> > +        }
> > +
> > +        /* Start of 'scope' */
> > +        if (strstart(map, "client", &map)) {
> > +            tmp_entry.flags |= XATTR_MAP_FLAG_CLIENT;
> > +        } else if (strstart(map, "server", &map)) {
> > +            tmp_entry.flags |= XATTR_MAP_FLAG_SERVER;
> > +        } else if (strstart(map, "all", &map)) {
> > +            tmp_entry.flags |= XATTR_MAP_FLAG_ALL;
> > +        } else {
> > +            fuse_log(FUSE_LOG_ERR,
> > +                     "%s: Unexpected scope;"
> > +                     " Expecting 'client', 'server', or 'all', in rule %zu\n",
> > +                     __func__, nentries);
> > +            exit(1);
> > +        }
> > +
> > +        if (*map++ != sep) {
> > +            fuse_log(FUSE_LOG_ERR,
> > +                     "%s: Expecting '%c' found '%c'"
> > +                     " after scope in rule %zu\n",
> > +                     __func__, sep, *map, nentries);
> > +            exit(1);
> > +        }
> > +
> > +        /* At start of 'key' field */
> > +        tmp = strchr(map, sep);
> > +        if (!tmp) {
> > +            fuse_log(FUSE_LOG_ERR,
> > +                     "%s: Missing '%c' at end of key field of rule %zu",
> > +                     __func__, sep, nentries);
> > +            exit(1);
> > +        }
> > +        tmp_entry.key = g_strndup(map, tmp - map);
> > +        map = tmp + 1;
> > +
> > +        /* At start of 'prepend' field */
> > +        tmp = strchr(map, sep);
> > +        if (!tmp) {
> > +            fuse_log(FUSE_LOG_ERR,
> > +                     "%s: Missing '%c' at end of prepend field of rule %zu",
> > +                     __func__, sep, nentries);
> > +            exit(1);
> > +        }
> > +        tmp_entry.prepend = g_strndup(map, tmp - map);
> > +        map = tmp + 1;
> > +
> > +        lo->xattr_map_list = add_xattrmap_entry(lo->xattr_map_list, &nentries,
> > +                                                &tmp_entry);
> > +        /* End of rule - go around again for another rule */
> > +    }
> > +
> > +    if (!nentries) {
> > +        fuse_log(FUSE_LOG_ERR, "Empty xattr map\n");
> > +        exit(1);
> > +    }
> > +
> > +    /* Add a terminator to error in cases the user hasn't specified */
> > +    tmp_entry.flags = XATTR_MAP_FLAG_ALL | XATTR_MAP_FLAG_END_BAD |
> > +                      XATTR_MAP_FLAG_LAST;
> > +    tmp_entry.key = g_strdup("");
> > +    tmp_entry.prepend = g_strdup("");
> > +    lo->xattr_map_list = add_xattrmap_entry(lo->xattr_map_list, &nentries,
> > +                                            &tmp_entry);
> 
> Not sure why this default rule is needed when user has not specified one.
> 
> This seems to be equivalent of ":bad:all:::". Will this not block all
> the xattrs which have not been caught by previous rules. And user
> probably did not want it. 

I might be able to get rid of that now;  my preference is to tell the
users they should be explicit about what happens.

Dave

> Thanks
> Vivek
>