README.fuchsia - third_party/yasm - Git at Google

 Name: yasm
 URL: http://www.tortall.net/projects/yasm/
 Upstream git: https://github.com/yasm/yasm
 Version: 1.3.0
 License: 2-clause or 3-clause BSD licensed, with the exception of bitvect, which is triple-licensed under the Artistic license, GPL, and LGPL
 License File: COPYING
 License Android Compatible: yes
 Security Critical: no
 Description:

 Yasm is a complete rewrite of the NASM assembler supporting the
 x86 and AMD64 instruction sets.

 Changes from upstream:

 * remove unneeded files
   rm -r  ConfigureChecks.cmake HACKING INSTALL ChangeLog \
       cmake tests README NEWS \
       Mkfiles YASM-VERSION-GEN.* autogen.sh \
       modules/arch/x86/tests \
       modules/dbgfmts/dwarf2/tests \
       modules/objfmts/elf/tests \
       modules/objfmts/macho/tests \
       modules/objfmts/win32/tests \
       modules/objfmts/win64/tests \
       modules/parsers/gas/tests \
       modules/parsers/nasm/tests \
       libyasm-stdint.h.cmake \
       config.h.cmake \
       configure.ac \
       Makefile.am \
       config/config.* config/mkinstalldirs \
       po/Makevars .gitignore
   find . -name CMakeLists.txt -delete
   find . -name Makefile* -delete
 * add files BUILD.gn, README.fuchsia, run_yasm.py, and yasm_assemble.gni
 * add folders config/linux and config/mac
 * modify frontends/tasm/tasm.c and frontends/yasm/yasm.c to make the builds deterministic
 * modify libyasm/genmodule.c and modules/arch/x86/gen_x86_insn.py to take an output file as a param
 * modify tools/genperf/perfect.c to comment out a stderr message (to not pollute build logs)
 * modify tools/re2c/code.c to fix a warning


 The BUILD.gn is adapted from
 https://chromium.googlesource.com/chromium/src/+/master/build/secondary/third_party/libjpeg_turbo/BUILD.gn

 which in turn is based on an deprecated yasm.gyp file (which has been deleted). At the bottom
 of this file, there are some notes from the deprecated yasm.gyp file about the yasm build process.

 Instructions for recreating the yasm.gyp file.
   1) Get a clean version of the yasm source tree.

   2) Run configure on the pristine source from a different directory (eg.,
      /tmp/yasm_build).  Running configure from another directory will keep
      the source tree clean.

   3) Next, capture all the output from a build of yasm.  We will use the build
      log as a reference for making the yasm.gyp file.

        make yasm > yasm_build_log 2> yasm_build_err

   4) Check yasm_build_err to see if there are any anomalies beyond yasm's
      compiler warnings.

   5) Grab the generated Makefile, libyasm-stdint.h, config.h, and put into
      the correct platform location. For android platform, copy the files
      generated for linux, but make sure that ENABLE_NLS is not defined to
      allow mac host compiles to work.  For ios, copy the files from mac.

        src/third_party/yasm/source/config/[platform]

      While we do not directly use the "Makefile" to build, it is needed by
      the "genmodule" subprogram as input for creating the available modules
      list.

   6) Make sure all the subprograms are represented in yasm.gyp.

        grep '^gcc' yasm_build_log  |
        grep -v ' -DHAVE_CONFIG_H '

      The yasm build creates a bunch of subprograms that in-turn generate
      more .c files in the build. Luckily the commands to generate the
      subprogram do not have -DHAVE_CONFIG_H as a cflag.

      From this list, make sure all the subprograms that are build have
      appropriate targets in the yasm.gyp.

      You will notice, when you get to the next step, that there are some
      .c source files that are compiled both for yasm, and for genperf.

      Those should go into the genperf_libs target so that they can be
      shared by the genperf and yasm targets.  Find those files by appending

        | grep 'gp-'

      to the command above.

   7) Find all the source files used to build yasm proper.

        grep -E '^gcc' yasm_build_log  |
        grep ' -DHAVE_CONFIG_H ' |
        awk '{print $NF }' |
        sed -e "s/'\.\/'\`//" |  # Removes some garbage from the build line.
        sort -u |
        sed -e "s/\(.*\)/'\1',/"  # Add quotes to each line.

      Reversing the -DHAVE_CONFIG_H filter from the command above should
      list the compile lines for yasm proper.

      This should get you close, but you will need to manually examine this
      list.  However, some of the built products are still included in the
      command above.  Generally, if the source file is in the root directory,
      it's a generated file.

      Inspect the current yasm.gyp for a list of the subprograms and their
      outputs.

      Update the sources list in the yasm target accordingly.  Read step #9
      as well if you update the source list to avoid problems.

   8) Update the actions for each of the subprograms.

      Here is the real fun.  For each subprogram created, you will need to
      update the actions and rules in yasm.gyp that invoke the subprogram to
      generate the files needed by the rest of the build.

      I don't have any good succinct instructions for this.  Grep the build
      log for each subprogram invocation (eg., "./genversion"), look at
      its command inputs and output, then verify our yasm.gyp does something
      similar.

      The good news is things likely only link or compile if this is done
      right so you'll know if there is a problem.

      Again, refer to the existing yasm.gyp for a guide to how the generated
      files are used.

      Here are a few gotchas:
        1) genmodule, by default, writes module.c into the current
           directory.  This does not play nicely with gyp.  We patch the
           source during build to allow specifying a specific output file.

        2) Most of the generated files, even though they are .c files, are
           #included by other files in the build.  Make sure they end up
           in a directory that is in the include path for the build.
           One of <(shared_generated_dir) or <(generated_dir) should work.

        3) Some of the genperf output is #included while others need to be
           compiled directly.  That is why there are 2 different rules for
           .gperf files in two targets.

   9) Check for python scripts that are run.

        grep python yasm_build_log

      Yasm uses python scripts to generate the assembly code description
      files in C++. Make sure to get these put into the gyp file properly as
      well.  An example is gen_x86_insn.py for x86 assembly.

      Note that at least the gen_x86_insn.py script suffers from the same
      problem as genmacro in that it outputs to the current directory by
      default.  The yasm.gyp build patches this file before invoking it to
      allow specifying an output directory.

  10) Recreate the 'AdditionalOptions!': [ '/analyze' ] block so that VC++
      /analyze builds won't fail.

  11) If all that's is finished, attempt to build....and cross your fingers.

 Notes from the deprecated yasm.gyp file:
 #
 # There are three classes of dynamically generated files:
 #   1) C source files that should be included in the build (eg., lc3bid.c)
 #   2) C source files that are #included by static C sources (eg., license.c)
 #   3) Intermediate files that are used as input by other subprograms to
 #      further generate files in category #1 or #2.  (eg., version.mac)
 #
 # This structure is represented with the following targets:
 #   1) yasm -- Sources, flags for the main yasm executable. Also has most of
 #              of the actions and rules that invoke the subprograms.
 #   2) config_sources -- Checked in version of files generated by manually
 #                        running configure that are used by all binaries.
 #   3) generate_files -- Actions and rules for files of type #3.
 #   4) genperf_libs -- Object files shared between yasm and the genperf
 #                      subprogram.
 #   5) genmacro, genmodule, etc. -- One executable target for each subprogram.
 #
 # You will notice that a lot of the action targets seem very similar --
 # especially for genmacro invocations. This makes it seem like they should
 # be a rule. The problem is that the correct invocation cannot be inferred
 # purely from the file name, or extension.  Nor is it obvious whether the
 # output should be processed as a source or not.  Thus, we are left with a
 # large amount of repetitive code.
	Name: yasm
	URL: http://www.tortall.net/projects/yasm/
	Upstream git: https://github.com/yasm/yasm
	Version: 1.3.0
	License: 2-clause or 3-clause BSD licensed, with the exception of bitvect, which is triple-licensed under the Artistic license, GPL, and LGPL
	License File: COPYING
	License Android Compatible: yes
	Security Critical: no
	Description:

	Yasm is a complete rewrite of the NASM assembler supporting the
	x86 and AMD64 instruction sets.

	Changes from upstream:

	* remove unneeded files
	rm -r ConfigureChecks.cmake HACKING INSTALL ChangeLog \
	cmake tests README NEWS \
	Mkfiles YASM-VERSION-GEN.* autogen.sh \
	modules/arch/x86/tests \
	modules/dbgfmts/dwarf2/tests \
	modules/objfmts/elf/tests \
	modules/objfmts/macho/tests \
	modules/objfmts/win32/tests \
	modules/objfmts/win64/tests \
	modules/parsers/gas/tests \
	modules/parsers/nasm/tests \
	libyasm-stdint.h.cmake \
	config.h.cmake \
	configure.ac \
	Makefile.am \
	config/config.* config/mkinstalldirs \
	po/Makevars .gitignore
	find . -name CMakeLists.txt -delete
	find . -name Makefile* -delete
	* add files BUILD.gn, README.fuchsia, run_yasm.py, and yasm_assemble.gni
	* add folders config/linux and config/mac
	* modify frontends/tasm/tasm.c and frontends/yasm/yasm.c to make the builds deterministic
	* modify libyasm/genmodule.c and modules/arch/x86/gen_x86_insn.py to take an output file as a param
	* modify tools/genperf/perfect.c to comment out a stderr message (to not pollute build logs)
	* modify tools/re2c/code.c to fix a warning


	The BUILD.gn is adapted from
	https://chromium.googlesource.com/chromium/src/+/master/build/secondary/third_party/libjpeg_turbo/BUILD.gn

	which in turn is based on an deprecated yasm.gyp file (which has been deleted). At the bottom
	of this file, there are some notes from the deprecated yasm.gyp file about the yasm build process.

	Instructions for recreating the yasm.gyp file.
	1) Get a clean version of the yasm source tree.

	2) Run configure on the pristine source from a different directory (eg.,
	/tmp/yasm_build). Running configure from another directory will keep
	the source tree clean.

	3) Next, capture all the output from a build of yasm. We will use the build
	log as a reference for making the yasm.gyp file.

	make yasm > yasm_build_log 2> yasm_build_err

	4) Check yasm_build_err to see if there are any anomalies beyond yasm's
	compiler warnings.

	5) Grab the generated Makefile, libyasm-stdint.h, config.h, and put into
	the correct platform location. For android platform, copy the files
	generated for linux, but make sure that ENABLE_NLS is not defined to
	allow mac host compiles to work. For ios, copy the files from mac.

	src/third_party/yasm/source/config/[platform]

	While we do not directly use the "Makefile" to build, it is needed by
	the "genmodule" subprogram as input for creating the available modules
	list.

	6) Make sure all the subprograms are represented in yasm.gyp.

	grep '^gcc' yasm_build_log \|
	grep -v ' -DHAVE_CONFIG_H '

	The yasm build creates a bunch of subprograms that in-turn generate
	more .c files in the build. Luckily the commands to generate the
	subprogram do not have -DHAVE_CONFIG_H as a cflag.

	From this list, make sure all the subprograms that are build have
	appropriate targets in the yasm.gyp.

	You will notice, when you get to the next step, that there are some
	.c source files that are compiled both for yasm, and for genperf.

	Those should go into the genperf_libs target so that they can be
	shared by the genperf and yasm targets. Find those files by appending

	\| grep 'gp-'

	to the command above.

	7) Find all the source files used to build yasm proper.

	grep -E '^gcc' yasm_build_log \|
	grep ' -DHAVE_CONFIG_H ' \|
	awk '{print $NF }' \|
	sed -e "s/'\.\/'\`//" \| # Removes some garbage from the build line.
	sort -u \|
	sed -e "s/\(.*\)/'\1',/" # Add quotes to each line.

	Reversing the -DHAVE_CONFIG_H filter from the command above should
	list the compile lines for yasm proper.

	This should get you close, but you will need to manually examine this
	list. However, some of the built products are still included in the
	command above. Generally, if the source file is in the root directory,
	it's a generated file.

	Inspect the current yasm.gyp for a list of the subprograms and their
	outputs.

	Update the sources list in the yasm target accordingly. Read step #9
	as well if you update the source list to avoid problems.

	8) Update the actions for each of the subprograms.

	Here is the real fun. For each subprogram created, you will need to
	update the actions and rules in yasm.gyp that invoke the subprogram to
	generate the files needed by the rest of the build.

	I don't have any good succinct instructions for this. Grep the build
	log for each subprogram invocation (eg., "./genversion"), look at
	its command inputs and output, then verify our yasm.gyp does something
	similar.

	The good news is things likely only link or compile if this is done
	right so you'll know if there is a problem.

	Again, refer to the existing yasm.gyp for a guide to how the generated
	files are used.

	Here are a few gotchas:
	1) genmodule, by default, writes module.c into the current
	directory. This does not play nicely with gyp. We patch the
	source during build to allow specifying a specific output file.

	2) Most of the generated files, even though they are .c files, are
	#included by other files in the build. Make sure they end up
	in a directory that is in the include path for the build.
	One of <(shared_generated_dir) or <(generated_dir) should work.

	3) Some of the genperf output is #included while others need to be
	compiled directly. That is why there are 2 different rules for
	.gperf files in two targets.

	9) Check for python scripts that are run.

	grep python yasm_build_log

	Yasm uses python scripts to generate the assembly code description
	files in C++. Make sure to get these put into the gyp file properly as
	well. An example is gen_x86_insn.py for x86 assembly.

	Note that at least the gen_x86_insn.py script suffers from the same
	problem as genmacro in that it outputs to the current directory by
	default. The yasm.gyp build patches this file before invoking it to
	allow specifying an output directory.

	10) Recreate the 'AdditionalOptions!': [ '/analyze' ] block so that VC++
	/analyze builds won't fail.

	11) If all that's is finished, attempt to build....and cross your fingers.

	Notes from the deprecated yasm.gyp file:
	#
	# There are three classes of dynamically generated files:
	# 1) C source files that should be included in the build (eg., lc3bid.c)
	# 2) C source files that are #included by static C sources (eg., license.c)
	# 3) Intermediate files that are used as input by other subprograms to
	# further generate files in category #1 or #2. (eg., version.mac)
	#
	# This structure is represented with the following targets:
	# 1) yasm -- Sources, flags for the main yasm executable. Also has most of
	# of the actions and rules that invoke the subprograms.
	# 2) config_sources -- Checked in version of files generated by manually
	# running configure that are used by all binaries.
	# 3) generate_files -- Actions and rules for files of type #3.
	# 4) genperf_libs -- Object files shared between yasm and the genperf
	# subprogram.
	# 5) genmacro, genmodule, etc. -- One executable target for each subprogram.
	#
	# You will notice that a lot of the action targets seem very similar --
	# especially for genmacro invocations. This makes it seem like they should
	# be a rule. The problem is that the correct invocation cannot be inferred
	# purely from the file name, or extension. Nor is it obvious whether the
	# output should be processed as a source or not. Thus, we are left with a
	# large amount of repetitive code.