Project: Part2 – Initial build testing on argon2 package using C and Assembler language(Progress 2)

Following the last blog called “Project: Part2 – Initial build testing on argon2 package using C and Assembler language(Progress 2 – Ongoing)

I will continue to blog about the build of argon2 password hashing function.

Testing Requirements

I will be testing using the following criteria:

  • Check how many inline assembler code already exist
  • Check how many dedicated/ separate assembler files exist in the package (File Extension: s or S)
  • Check how many files use the C programming language (File Extension: c)
  • Use a profiling tool to check how optimized the program is in the current build package (Tools such as: gprof, stap, etc.)
  • Build the package and check the default results by testing the build against a password file. The file will be made using a Microsoft Excel formula.
  • Check that the results are relatively consistent. (This will be from program compile time, program run time, and the program file size)
  • Introduce a minor or major change that will optimize or reduce performance of the original program. This is done by changing the character(s) of the password file generated by the Microsoft Excel formula.
  • Compare the results of the changes and the original program results.

Tasks:

Check how many inline assembler code already exist:

This is to check if the package, (Example: argon2), has any built-in assembler language code to optimize the program.

The following is the command to check:

egrep "__asm__" -R

The command egrep is a feature of Linux that allows for searching specific characters within files. Having the -R option will change the search requirement to also search recursively (Searches all sub-directories).

This was my output:

src/bench.c: __asm__ __volatile__("rdtsc" : "=a"(rax), "=d"(rdx) : :);
src/bench.c: __asm__ __volatile__("rdtsc" : "=A"(rax) : :);

This shows a file with two inline assembler codes within the directory /home/username/phc-winner-argon2/src/build.c

Check how many dedicated/ separate assembler files exist in the package (File Extension: s or S)

We know that there are only two lines of inline assembler code within the file bench.c but not having separate assembler files will either mean the program does not need to interact with the hardware or the program will not need assembler code to be optimized.

The following code:

find . -name "*.[sS]"

The find command is a feature in Linux that allows us to search for files and directories instead of text within a file. The -name will filter by the file name and the “*.[sS]” is the regular expression for filtering for specific file extensions that end in a .s or .S.

There was no files found as the result nothing.

Check how many files use the C programming language (File Extension: c)

I will now check for how many files are written in the C programming language for argon2. I will be using the find command again but with a different regular expression to check for file extensions ending in a .c.

This is the command:

find . -name "*.c"

This is the output:

./src/argon2.c
./src/bench.c
./src/blake2/blake2b.c
./src/core.c
./src/encoding.c
./src/genkat.c
./src/opt.c
./src/ref.c
./src/run.c
./src/test.c
./src/thread.c

Counting the number of files require us to use the pipe command “|”

Like this:

find . -name "*.c" | wc -l

The pipe command will format the left side of the “|” into the format required on the right side command. The right side command, wc -l, performs a count of the lines using the option -l. This command is handy for counting a lot of files in a short amount of time.

The result was 11 files.

Building/ Testing the original argon2 package

In the previous post, I started in the directory of /home/username/projects/phc-winner-argon2/src.

First Attempt

The first attempt to build the argon2 program using the following command:

gcc -o argon2 argon2.c

This did not work as the header file “argon2.h” was missing.

The header was actually located in the include directory in the following path, for me: /home/username/phc-winner-argon2/include.

I copied the file using this command:

cp ../include/argon2.h .

I then rebuild having the header file in the directory /home/username/phc-winner-argon2/src using the previous command:

gcc -o argon2 argon2.c

This was the result I got:

[dcchen@aarchie src]$ gcc -o argon2 argon2.c
/usr/lib/gcc/aarch64-redhat-linux/8/../../../../lib64/crt1.o: In function `_start':
(.text+0x18): undefined reference to `main'
(.text+0x1c): undefined reference to `main'
(.text+0x20): undefined reference to `main'
(.text+0x24): undefined reference to `main'
/tmp/ccMi4kO3.o: In function `argon2_ctx':
argon2.c:(.text+0xc0): undefined reference to `validate_inputs'
argon2.c:(.text+0x1e4): undefined reference to `initialize'
argon2.c:(.text+0x204): undefined reference to `fill_memory_blocks'
argon2.c:(.text+0x22c): undefined reference to `finalize'
/tmp/ccMi4kO3.o: In function `argon2_hash':
argon2.c:(.text+0x378): undefined reference to `clear_internal_memory'
argon2.c:(.text+0x3d4): undefined reference to `encode_string'
argon2.c:(.text+0x3e8): undefined reference to `clear_internal_memory'
argon2.c:(.text+0x3f4): undefined reference to `clear_internal_memory'
argon2.c:(.text+0x410): undefined reference to `clear_internal_memory'
/tmp/ccMi4kO3.o: In function `argon2_verify':
argon2.c:(.text+0x890): undefined reference to `decode_string'
/tmp/ccMi4kO3.o: In function `argon2_encodedlen':
argon2.c:(.text+0xd24): undefined reference to `numlen'
argon2.c:(.text+0xd30): undefined reference to `numlen'
argon2.c:(.text+0xd3c): undefined reference to `numlen'
argon2.c:(.text+0xd48): undefined reference to `b64len'
argon2.c:(.text+0xd54): undefined reference to `b64len'
argon2.c:(.text+0xd60): undefined reference to `numlen'
collect2: error: ld returned 1 exit status

This did not build.

Second Attempt

The second attempt is from using the Makefile that was included in the argon2 package.

The following is the Makefile file located in /home/username/projects/phc-winner-argon2/Makefile.

Note: This can be view/edit by Vim Editor or whichever Linux text editor.

#
# Argon2 reference source code package - reference C implementations
#
# Copyright 2015
# Daniel Dinu, Dmitry Khovratovich, Jean-Philippe Aumasson, and Samuel Neves
#
# You may use this work under the terms of a Creative Commons CC0 1.0
# License/Waiver or the Apache Public License 2.0, at your option. The terms of
# these licenses can be found at:
#
# - CC0 1.0 Universal : http://creativecommons.org/publicdomain/zero/1.0
# - Apache 2.0 : http://www.apache.org/licenses/LICENSE-2.0
#
# You should have received a copy of both of these licenses along with this
# software. If not, they may be obtained at the above URLs.
#

RUN = argon2
BENCH = bench
GENKAT = genkat

# Increment on an ABI breaking change
ABI_VERSION = 1

DIST = phc-winner-argon2

SRC = src/argon2.c src/core.c src/blake2/blake2b.c src/thread.c src/encoding.c
SRC_RUN = src/run.c
SRC_BENCH = src/bench.c
SRC_GENKAT = src/genkat.c
OBJ = $(SRC:.c=.o)

CFLAGS += -std=c89 -O3 -Wall -g -Iinclude -Isrc

ifeq ($(NO_THREADS), 1)
CFLAGS += -DARGON2_NO_THREADS
else
CFLAGS += -pthread
endif

CI_CFLAGS := $(CFLAGS) -Werror=declaration-after-statement -D_FORTIFY_SOURCE=2 \
-Wextra -Wno-type-limits -Werror -coverage -DTEST_LARGE_RAM

OPTTARGET ?= native
OPTTEST := $(shell $(CC) -Iinclude -Isrc -march=$(OPTTARGET) src/opt.c -c \
-o /dev/null 2>/dev/null; echo $$?)
# Detect compatible platform
ifneq ($(OPTTEST), 0)
$(info Building without optimizations)
SRC += src/ref.c
else
$(info Building with optimizations for $(OPTTARGET))
CFLAGS += -march=$(OPTTARGET)
SRC += src/opt.c
endif

BUILD_PATH := $(shell pwd)
KERNEL_NAME := $(shell uname -s)

LIB_NAME=argon2
ifeq ($(KERNEL_NAME), Linux)
LIB_EXT := so.$(ABI_VERSION)
LIB_CFLAGS := -shared -fPIC -fvisibility=hidden -DA2_VISCTL=1
SO_LDFLAGS := -Wl,-soname,lib$(LIB_NAME).$(LIB_EXT)
LINKED_LIB_EXT := so
endif
ifeq ($(KERNEL_NAME), $(filter $(KERNEL_NAME),FreeBSD NetBSD OpenBSD))
LIB_EXT := so
LIB_CFLAGS := -shared -fPIC
endif
ifeq ($(KERNEL_NAME), Darwin)
LIB_EXT := $(ABI_VERSION).dylib
LIB_CFLAGS := -dynamiclib -install_name @rpath/lib$(LIB_NAME).$(LIB_EXT)
LINKED_LIB_EXT := dylib
endif
ifeq ($(findstring CYGWIN, $(KERNEL_NAME)), CYGWIN)
LIB_EXT := dll
LIB_CFLAGS := -shared -Wl,--out-implib,lib$(LIB_NAME).$(LIB_EXT).a
endif
ifeq ($(findstring MINGW, $(KERNEL_NAME)), MINGW)
LIB_EXT := dll
LIB_CFLAGS := -shared -Wl,--out-implib,lib$(LIB_NAME).$(LIB_EXT).a
endif
ifeq ($(findstring MSYS, $(KERNEL_NAME)), MSYS)
LIB_EXT := dll
LIB_CFLAGS := -shared -Wl,--out-implib,lib$(LIB_NAME).$(LIB_EXT).a
endif
ifeq ($(KERNEL_NAME), SunOS)
CC := gcc
CFLAGS += -D_REENTRANT
LIB_EXT := so
LIB_CFLAGS := -shared -fPIC
endif

ifeq ($(KERNEL_NAME), Linux)
ifeq ($(CC), clang)
CI_CFLAGS += -fsanitize=address -fsanitize=undefined
endif
endif

LIB_SH := lib$(LIB_NAME).$(LIB_EXT)
LIB_ST := lib$(LIB_NAME).a

ifdef LINKED_LIB_EXT
LINKED_LIB_SH := lib$(LIB_NAME).$(LINKED_LIB_EXT)
endif


LIBRARIES = $(LIB_SH) $(LIB_ST)
HEADERS = include/argon2.h

INSTALL = install

DESTDIR =
PREFIX = /usr
INCLUDE_REL = include
LIBRARY_REL = lib
BINARY_REL = bin

INST_INCLUDE = $(DESTDIR)$(PREFIX)/$(INCLUDE_REL)
INST_LIBRARY = $(DESTDIR)$(PREFIX)/$(LIBRARY_REL)
INST_BINARY = $(DESTDIR)$(PREFIX)/$(BINARY_REL)

.PHONY: clean dist format $(GENKAT) all install

all: $(RUN) libs
libs: $(LIBRARIES)

$(RUN): $(SRC) $(SRC_RUN)
$(CC) $(CFLAGS) $(LDFLAGS) $^ -o $@

$(BENCH): $(SRC) $(SRC_BENCH)
$(CC) $(CFLAGS) $^ -o $@

$(GENKAT): $(SRC) $(SRC_GENKAT)
$(CC) $(CFLAGS) $^ -o $@ -DGENKAT

$(LIB_SH): $(SRC)
$(CC) $(CFLAGS) $(LIB_CFLAGS) $(LDFLAGS) $(SO_LDFLAGS) $^ -o $@

$(LIB_ST): $(OBJ)
ar rcs $@ $^

clean:
rm -f $(RUN) $(BENCH) $(GENKAT)
rm -f $(LIB_SH) $(LIB_ST) kat-argon2*
rm -f testcase
rm -rf *.dSYM
cd src/ && rm -f *.o
cd src/blake2/ && rm -f *.o
cd kats/ && rm -f kat-* diff* run_* make_*

dist:
cd ..; \
tar -c --exclude='.??*' -z -f $(DIST)-`date "+%Y%m%d"`.tgz $(DIST)/*

test: $(SRC) src/test.c
$(CC) $(CFLAGS) -Wextra -Wno-type-limits $^ -o testcase
@sh kats/test.sh
./testcase

testci: $(SRC) src/test.c
$(CC) $(CI_CFLAGS) $^ -o testcase
@sh kats/test.sh
./testcase

.PHONY: test

format:
clang-format -style="{BasedOnStyle: llvm, IndentWidth: 4}" \
-i include/*.h src/*.c src/*.h src/blake2/*.c src/blake2/*.h

install: $(RUN) libs
$(INSTALL) -d $(INST_INCLUDE)
$(INSTALL) -m 0644 $(HEADERS) $(INST_INCLUDE)
$(INSTALL) -d $(INST_LIBRARY)
$(INSTALL) $(LIBRARIES) $(INST_LIBRARY)
ifdef LINKED_LIB_SH
cd $(INST_LIBRARY) && ln -s $(notdir $(LIB_SH) $(LINKED_LIB_SH))
endif
$(INSTALL) -d $(INST_BINARY)
$(INSTALL) $(RUN) $(INST_BINARY)

uninstall:
cd $(INST_INCLUDE) && rm -f $(notdir $(HEADERS))
cd $(INST_LIBRARY) && rm -f $(notdir $(LIBRARIES) $(LINKED_LIB_SH))
cd $(INST_BINARY) && rm -f $(notdir $(RUN))

I noticed that the following code was in the Makefile:

SRC = src/argon2.c src/core.c src/blake2/blake2b.c src/thread.c src/encoding.c
SRC_RUN = src/run.c
SRC_BENCH = src/bench.c
SRC_GENKAT = src/genkat.c
OBJ = $(SRC:.c=.o)

CFLAGS += -std=c89 -O3 -Wall -g -Iinclude -Isrc

The Makefile all the links to the required header and source files as seen in red.

I changed my current directory from /home/username/projects/phc-winner-argon2/src to /home/username/projects/phc-winner-argon2/ using the following command:

cd /home/username/projects/phc-winner-argon2

I ran the make command to build the program.

This is the command:

make argon2

I will instead run this command instead to check the time it takes to build the program:

time make argon2

This was the result:

real    0m2.519s
user    0m2.180s
sys     0m0.327s

Real is the time combining the user inputting the command and the system time to execute the command. The user time is the time it took to input the command. The sys time is the time it took to execute the command.

(Will continue in Part 2 – Progress 3)

 

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s