[dpdk-dev] [PATCH] pmdinfogen: allow using stdin and stdout

[dpdk-dev] [PATCH] pmdinfogen: allow using stdin and stdout

[Bruce Richardson]

Rather than having to work off files all the time, allow stdin and stdout
to be used as the source and destination for pmdinfogen. This will allow
other possible usages from scripts, e.g. taking files from ar archive and
building a single .pmd.c file from all the .o files in it.

	for f in `ar t librte_pmd_xyz.a` ; do
		ar p librte_pmd_xyz.a $f | pmdinfogen - - >> xyz_info.c
	done

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 buildtools/pmdinfogen/pmdinfogen.c | 44 +++++++++++++++++++++++++++++++-------
 1 file changed, 36 insertions(+), 8 deletions(-)

diff --git a/buildtools/pmdinfogen/pmdinfogen.c b/buildtools/pmdinfogen/pmdinfogen.c
index b07dbcf4b..45b267346 100644
--- a/buildtools/pmdinfogen/pmdinfogen.c
+++ b/buildtools/pmdinfogen/pmdinfogen.c
@@ -26,6 +26,7 @@
 #define ADDR_SIZE 32
 #endif
 
+static int use_stdin, use_stdout;
 
 static const char *sym_name(struct elf_info *elf, Elf_Sym *sym)
 {
@@ -39,11 +40,32 @@ static void *grab_file(const char *filename, unsigned long *size)
 {
 	struct stat st;
 	void *map = MAP_FAILED;
-	int fd;
+	int fd = -1;
+
+	if (!use_stdin) {
+		fd = open(filename, O_RDONLY);
+		if (fd < 0)
+			return NULL;
+	} else {
+		/* from stdin, use a temporary file to mmap */
+		FILE *infile;
+		char buffer[1024];
+		size_t n;
+
+		infile = tmpfile();
+		if (infile == NULL) {
+			perror("tmpfile");
+			return NULL;
+		}
+		while (!feof(stdin)) {
+			n = fread(buffer, 1, sizeof(buffer), stdin);
+			if (fwrite(buffer, 1, n, infile) != n)
+				goto failed;
+		}
+		fflush(infile);
+		fd = fileno(infile);
+	}
 
-	fd = open(filename, O_RDONLY);
-	if (fd < 0)
-		return NULL;
 	if (fstat(fd, &st))
 		goto failed;
 
@@ -358,10 +380,14 @@ static void output_pmd_info_string(struct elf_info *info, char *outfile)
 	struct rte_pci_id *pci_ids;
 	int idx = 0;
 
-	ofd = fopen(outfile, "w+");
-	if (!ofd) {
-		fprintf(stderr, "Unable to open output file\n");
-		return;
+	if (use_stdout)
+		ofd = stdout;
+	else {
+		ofd = fopen(outfile, "w+");
+		if (!ofd) {
+			fprintf(stderr, "Unable to open output file\n");
+			return;
+		}
 	}
 
 	drv = info->drivers;
@@ -411,6 +437,8 @@ int main(int argc, char **argv)
 			basename(argv[0]));
 		exit(127);
 	}
+	use_stdin = !strcmp(argv[1], "-");
+	use_stdout = !strcmp(argv[2], "-");
 	parse_elf(&info, argv[1]);
 
 	if (locate_pmd_entries(&info) < 0)
-- 
2.14.3

Re: [dpdk-dev] [PATCH] pmdinfogen: allow using stdin and stdout

[Neil Horman]

On Thu, Jan 25, 2018 at 11:12:25AM +0000, Bruce Richardson wrote:
> Rather than having to work off files all the time, allow stdin and stdout
> to be used as the source and destination for pmdinfogen. This will allow
> other possible usages from scripts, e.g. taking files from ar archive and
> building a single .pmd.c file from all the .o files in it.
> 
> 	for f in `ar t librte_pmd_xyz.a` ; do
> 		ar p librte_pmd_xyz.a $f | pmdinfogen - - >> xyz_info.c
> 	done
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
I think this is a great idea.  FWIW, I think the ELF header at the front of the
file should have something that tells you the overall disk file size in it.  The
ELF header is pretty easy to parse and do some basic evaluation on, so you could
accelerate that read a bit, by breaking it down into a read of the size of the
header, followed by a parsing of it and a subsequent read of the overall size,
but that probably just nice to have

Either way, this is a worthwhile feature to have
Acked-by: Neil Horman <nhorman@tuxdriver.com>

Re: [dpdk-dev] [PATCH] pmdinfogen: allow using stdin and stdout

[Bruce Richardson]

On Thu, Jan 25, 2018 at 04:30:40PM -0500, Neil Horman wrote:
> On Thu, Jan 25, 2018 at 11:12:25AM +0000, Bruce Richardson wrote:
> > Rather than having to work off files all the time, allow stdin and
> > stdout to be used as the source and destination for pmdinfogen. This
> > will allow other possible usages from scripts, e.g. taking files
> > from ar archive and building a single .pmd.c file from all the .o
> > files in it.
> > 
> > 	for f in `ar t librte_pmd_xyz.a` ; do ar p librte_pmd_xyz.a $f |
> > 	pmdinfogen - - >> xyz_info.c done
> > 
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> I think this is a great idea.  FWIW, I think the ELF header at the
> front of the file should have something that tells you the overall
> disk file size in it.  The ELF header is pretty easy to parse and do
> some basic evaluation on, so you could accelerate that read a bit, by
> breaking it down into a read of the size of the header, followed by a
> parsing of it and a subsequent read of the overall size, but that
> probably just nice to have
> 
Yes, it's a good idea for future implementation. It would also allow
multiple .o files to be streamed together, since pmdinfogen could
recognise the end of one elf file and the start of another. That would
remove the need for the loop in the example above, since we could just
pipe the "ar p" output of the whole archive straight to pmdinfogen.

Not something for this release though. :-)

Thanks,
/Bruce

> Either way, this is a worthwhile feature to have
> Acked-by: Neil Horman <nhorman@tuxdriver.com>
> 

Re: [dpdk-dev] [PATCH] pmdinfogen: allow using stdin and stdout

[Thomas Monjalon]

25/01/2018 22:30, Neil Horman:
> On Thu, Jan 25, 2018 at 11:12:25AM +0000, Bruce Richardson wrote:
> > Rather than having to work off files all the time, allow stdin and stdout
> > to be used as the source and destination for pmdinfogen. This will allow
> > other possible usages from scripts, e.g. taking files from ar archive and
> > building a single .pmd.c file from all the .o files in it.
> > 
> > 	for f in `ar t librte_pmd_xyz.a` ; do
> > 		ar p librte_pmd_xyz.a $f | pmdinfogen - - >> xyz_info.c
> > 	done
> > 
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> I think this is a great idea.  FWIW, I think the ELF header at the front of the
> file should have something that tells you the overall disk file size in it.  The
> ELF header is pretty easy to parse and do some basic evaluation on, so you could
> accelerate that read a bit, by breaking it down into a read of the size of the
> header, followed by a parsing of it and a subsequent read of the overall size,
> but that probably just nice to have
> 
> Either way, this is a worthwhile feature to have
> Acked-by: Neil Horman <nhorman@tuxdriver.com>

Applied, thanks