Re: [dpdk-dev] [PATCH v6 02/11] eal: introduce internal wrappers for file operations

[Neil Horman <nhorman@tuxdriver.com>]

On Fri, Jun 05, 2020 at 03:16:03AM +0300, Dmitry Kozlyuk wrote:
> On Thu, 4 Jun 2020 17:07:07 -0400
> Neil Horman <nhorman@tuxdriver.com> wrote:
> 
> > On Wed, Jun 03, 2020 at 03:34:03PM +0300, Dmitry Kozlyuk wrote:
> > > On Wed, 3 Jun 2020 08:07:59 -0400
> > > Neil Horman <nhorman@tuxdriver.com> wrote:
> > > 
> > > [snip]  
> > > > > +int
> > > > > +eal_file_create(const char *path)
> > > > > +{
> > > > > +	int ret;
> > > > > +
> > > > > +	ret = open(path, O_CREAT | O_RDWR, 0600);
> > > > > +	if (ret < 0)
> > > > > +		rte_errno = errno;
> > > > > +
> > > > > +	return ret;
> > > > > +}
> > > > > +    
> > > > You don't need this call if you support the oflags option in the open call
> > > > below.  
> > > 
> > > See below.
> > >   
> > > > > +int
> > > > > +eal_file_open(const char *path, bool writable)
> > > > > +{
> > > > > +	int ret, flags;
> > > > > +
> > > > > +	flags = writable ? O_RDWR : O_RDONLY;
> > > > > +	ret = open(path, flags);
> > > > > +	if (ret < 0)
> > > > > +		rte_errno = errno;
> > > > > +
> > > > > +	return ret;
> > > > > +}
> > > > > +    
> > > > why are you changing this api from the posix file format (with oflags
> > > > specified).  As far as I can see both unix and windows platforms support that  
> > > 
> > > There is a number of caveats, which IMO make this approach better:
> > > 
> > > 1. Filesystem permissions on Windows are complicated. Supporting anything
> > > other than 0600 would add a lot of code, while EAL doesn't really need it.
> > > Microsoft's open() takes not permission bits, but a set of flags.
> > > 
> > > 2. Restricted interface prevents EAL developers from accidentally using
> > > features not supported on all platforms via a seemingly rich API.
> > > 
> > > 3. Microsoft CRT (the one Clang is using) deprecates open() in favor of
> > > _sopen_s() and issues a warning, and we're targeting -Werror. Disabling all
> > > such warnings (_CRT_SECURE_NO_DEPRECATE) doesn't seem right when CRT vendor
> > > encourages using alternatives. This is the primary reason for open()
> > > wrappers in v6.
> > >   
> > 
> > that seems a bit shortsighted to me.  By creating wrappers that restrict
> > functionality to the least common demoninator of supported platforms restricts
> > what all platforms are capable of.  For example, theres no reason that the eal
> > library shouldn't be able to open a file O_TRUNC or O_SYNC just because its
> > complex to do it on a single platform.  
> 
> The purpose of these wrappers is to maximize reuse of common code. It doesn't
> require POSIX par se, it's just implemented in terms of API that had been
> available on all supported OSes until Windows target was introduced. Wrapper
> interface is derived from common code requirements.
> 
Sure, and I'm fine with that.  What I'm concerned about is implementing wrappers
that define their APIs in terms of whats currently implemented.  Theres no
reason that the existing in use feature set won't be built upon in the future,
and the API should be able to handle that.

> > The API should be written to support the full range of functionality on all
> > platforms, and the individual implementations should write the code to make that
> > happen, or return an error that its unsupported on this particular platform.
> 
> IMO, common code, by definition, should avoid partial support of anything.
> 
I disagree.  Anytime you abstract an implementation to a more generic api, you
have the possibility that a given implementation won't offer full support for
all of the APIs features, and thats ok, as long as there are no users of the
features, and a given implmentation properly returns an error when their usage
is attempted.  The expectation then is, that the user of the feature will add
the feature to all implementations, so that the code can remain portable.  What
you shouldn't do is define the API such that those features can't be implemented
without having to change the API, as that runs the potential risk of having to
modify the ABI.  Thats probably not the case here, but the notion stands.  If
you write the API to encompass the superset of supported platforms features, the
rest is implementation details.

> > I'm not saying that you have to implement everything now, but you shouldn't
> > restrict the API from being able to do so in the future.  Otherwise, in the
> > future, if someone wants to implement O_TRUNC support (just to site an example),
> > they're going to have to make a change to the API above, and alter the
> > implementation for all the platforms anyway.  You may as well make the API
> > robust enough to support that now.
> 
> I agree that these particular wrappers can have a lot more options, so
> probably flags would be better. However, I wouldn't add parameters that
> have partial support, namely, permissions.
But windows does offer file and folder permissions:
https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-2000-server/bb727008(v=technet.10)?redirectedfrom=MSDN

As you said, they are complicated, and the security model is different, but
those are details that can be worked out/emulated when the need arises.

> What do you think of the following
> (names shortened)?
> 
> enum mode {
> 	RO = 0,	/* write-only is not portable */
> 	RW = 1,
> 	CREATE = 2	/* always 0600 equivalent */
> };
> 
> eal_file_open(const char *path, int mode);
> 
Yeah, that makes sense to me. I'd be good with that.

Neil

> -- 
> Dmitry Kozlyuk
>