First sketch for an easier configuration #301

czurnieden · 2019-05-28T21:51:32Z

A rough(!) sketch for a (half)automated configuration. Just list the things you want in make_config.conf and run make_config.pl to get the correct makefiles.
It still needs a make clean before the make or ar still gathers everything.

I tried to reuse as much as possible from helper.pl because such thing should be put there, but started with an isolated script to ease your judgment.

A make new_file or helper.pl --update-makefiles should return the original makefiles.

Updating MSVC project files is a todo, I don't know enough to do it safely.

make_config.pl

minad · 2019-05-29T08:11:30Z

Some comments/a wishlist:

This script should replace the generation in helper.pl (helper.pl --update-files) in the end as you said
A file tommath_config.h should be generated which contains a macro list of all activated functions and their transitive deps (flat list, not what we have in tommath_class.h)
tommath_class.h and tommath_superclass.h should be removed
maybe provide a different tommath.rsa.config and tommath.default.config (or put them in config/default, config/rsa etc), since this is what these superclasses are for
the #if guards in the c files could go away if we take the route via makefile generation. At least in the files with only a single function. Alternatively we keep them everywhere and don't update the makefiles based on the config. What do you think?
helper.pl --update-files should also produce an amalgamated version of tommath, tommath_amalgam.c (this is what gen.pl does). The amalgam should only contain the functions and deps selected in the configuration. Replace gen.pl which is used for profiling. In my private version I am also making functions static automatically.
maybe Get rid of BN prefix #284 should be merged first if we start reworking the build/config system (cleanup of file naming convention), add feature detection macro MP_HAS (2) #262 is also on my wishlist obviously
The perl can probably simplified, there is for example File::Slurper. But this could be done when we merge this with helper.pl
We need a fancy menuconfig gui using dialog --checklist ... which configures the config/default config/rsa etc ;)

sjaeckel · 2019-05-29T08:35:05Z

I don't understand the point of such a thing ... that's exactly why tommath_superclass.h exists

$ git diff -U1
diff --git a/tommath_superclass.h b/tommath_superclass.h
index cf17866..b355e0f 100644
--- a/tommath_superclass.h
+++ b/tommath_superclass.h
@@ -6,3 +6,8 @@
 /* default ... include all MPI */
-#define LTM_ALL
+//#define LTM_ALL
+#   define BN_CONVERSION_C
+#   define BN_CUTOFFS_C
+#   define BN_MP_MUL_C
+#   define BN_MP_SET_U32_C
+#   define BN_MP_GET_I32_C

$ cat demo/small.c
#include "tommath.h"

int main(int argc, char **argv)
{
   mp_int a, b;

   mp_init_multi(&a, &b, NULL);

   mp_set_u32(&a, 4u);
   mp_set_u32(&b, 4u);

   mp_mul(&a, &b, &a);

   return mp_get_u32(&a) != 16;
}

$ gcc demo/small.c -I. -L. -ltommath -o small
$ nm -C small | grep mp_
0000000000404de0 T mp_add
0000000000404e60 T mp_clamp
0000000000404fc0 T mp_clear
0000000000404ff0 T mp_clear_multi
00000000004050a0 T mp_cmp_mag
0000000000405300 T mp_div_2
00000000004055d0 T mp_div_3
0000000000405730 T mp_exch
00000000004009b0 T mp_get_i32
00000000004009d0 T mp_get_i64
0000000000400b10 T mp_get_mag32
0000000000400b20 T mp_get_mag64
0000000000405770 T mp_grow
0000000000400ef0 T mp_init
0000000000400cd0 T mp_init_i32
0000000000400e20 T mp_init_i64
0000000000400f30 T mp_init_multi
00000000004057e0 T mp_init_size
0000000000400c60 T mp_init_u32
0000000000400d80 T mp_init_u64
0000000000405830 T mp_lshd
00000000004011f0 T mp_mul
0000000000405be0 T mp_mul_2
0000000000400860 T mp_set_i32
00000000004008f0 T mp_set_i64
0000000000400780 T mp_set_u32
00000000004007e0 T mp_set_u64
0000000000405da0 T mp_sub
0000000000405e30 T s_mp_add
00000000004012f0 T s_mp_balance_mul
0000000000401a00 T s_mp_karatsuba_mul
0000000000402810 T s_mp_mul_digs
0000000000402b70 T s_mp_mul_digs_fast
0000000000402e90 T s_mp_sub
0000000000403380 T s_mp_toom_mul

sjaeckel · 2019-05-29T08:39:53Z

The only thing I could see is having a parser which is going through "your own sources"/the sources using libtommath and then spits out something that can be put in tommath_superclass.h

minad · 2019-05-29T09:22:23Z

For me the current mechanism is not sufficient at least. Then there are still some deficiencies like def file generation, issue with bn_conversion, bn_deprecated, no support for amalgam. I don't think the class mechanism is particularly nice but ymmv.

What czurnieden proposes is simpler. The downside however is that it needs a script.

sjaeckel · 2019-05-29T10:57:30Z

For me the current mechanism is not sufficient at least.

what exactly is missing?

Then there are still some deficiencies like def file generation, issue with bn_conversion, bn_deprecated

those are other issues, but yes they must be solved... btw. bn_conversion, bn_deprecated were also the only problems I had with the example ;)

no support for amalgam

the class mechanism supports that

What czurnieden proposes is simpler.

I don't see a difference in adding lines to a file called make_config.conf vs. adding lines to a file called tommath_superclass.h
... yeah one needs to know a minor subset of the syntax of the C preprocessor vs. only a list of function names, but meh... that's no real argument IMO

Maybe I also don't get the advantage of the proposed approach!? Please feel free to explain me what's different and better :-)

minad · 2019-05-29T11:23:09Z

no support for amalgam

the class mechanism supports that

Yes. But I would like to have it such that only used functions are included.
If I want that there is no way around a generator script though and the approach is different from the one taken by tommath_superclass. But sure, this is a matter of preference.

I don't see a difference in adding lines to a file called make_config.conf vs. adding lines to a file called tommath_superclass.h ... yeah one needs to know a minor subset of the syntax of the C preprocessor vs. only a list of function names, but meh... that's no real argument IMO

Sure, we could also stay with a file full of defines. I think here we are exploring what kind of alternative approach we could take. This will not necessarily end up in ltm. But if it turns out to be technically better, why not? As I said from my pov, I don't think the class/superclass mechanism is perfect.

However the important issues we are observering are due to the insufficient parsing in helper.pl. Besides the one-file-per-function issue of bn_conversion and bn_deprecated there is also another one: Basically mp_mul has a "weak dependency" on s_mp_mul_karatsuba. This should be reflected via the configuration such that mp_mul does not automatically require karatsuba. This could be parsed more easily given #262 since we could look for MP_HAS.

So instead of conflating two different issues here we could also split the discussion:

Fixing stuff: Dependency resolution (bn_conversion, bn_deprecated), weak dependencies, def-file generation. Furthermore the dep generation in helper.pl is still a mess (even after Simplify scripts #293). It does too many things at the same time:
1. dependency parsing
2. callgraph printing
3. class generation
4. rewriting files and adding #ifdef BN_X_C
New config system: Replacing the superclass/class mechanism with something better if desired

sjaeckel · 2019-05-29T11:34:51Z

So instead of conflating two different issues here we could also split the discussion:

Fine by me, should we merge #300 then and work on an improved version in another PR? or should we use #300 to already implement an improved version (not necessarily by @nijtmans)

As I didn't get an answer to my question "what is missing?" I have to guess from your comment... so you don't want to always build all files!? and only have the used files in the amalgamated version?!

minad · 2019-05-29T11:43:29Z

Fine by me, should we merge #300 then and work on an improved version in another PR? or should we use #300 to already implement an improved version (not necessarily by @nijtmans)

I suggest to merge it and we improve it afterwards. It is sufficient as a first step.

As I didn't get an answer to my question "what is missing?" I have to guess from your comment... so you don't want to always build all files!? and only have the used files in the amalgamated version?!

Yes. Not always building all files wasn't my idea, this is what @czurnieden came up with here. But for the amalgam, this is what I would like. And I need the tommath_class.h to be non buggy since it still includes too much, i.e., the weak dependencies.

czurnieden · 2019-05-29T11:56:17Z

(BIt late for me to chime in, hu?)

I don't understand the point of such a thing ... that's exactly why tommath_superclass.h exists

The main advantage of the tommath_class.h-tommath_superclass.h is that you can manipulate it manually. Actually, it is probably the single advantage and only if you don't have Perl and are dependent on a pre-generated tommath_class.h.

The script combines (sort of) tommath_class.h and tommath_superclass.h and uses the intersection. You can use the result as preprocessor directives in a file tommath_please_compile_only_these_functions_into_the_lib.h replacing the tommath_class.h-tommath_superclass.h combination or manipulate the makefiles directly.

The first has no real advantage over the old system, the default for LTM would still be to deliver a complete set in tommath_please_compile_only_these_functions_into_the_lib.h and you have a bit less functionality, too, because you cannot do it manually anymore, at least not easily.

I don't see a difference in adding lines to a file called make_config.conf vs. adding lines to a file called tommath_superclass.h

The file make_config.conf was used to keep the example as simple as possible. The information needed can be generated in many different ways, not necessarily a flat file with manually entered entries.

Doing it by manipulating the makefiles directly would make it possible to get rid of all the preprocessor stuff that tried to replace what autoconf/cmake does anywhere else.

Some branchings in LTM's code rely on these directives and would need to get changed. They are to be found in the filesbn_conversion.c, bn_deprecated.c, bn_mp_div.c, bn_mp_div_d.c, bn_mp_exptmod.c, bn_mp_invmod.c, bn_s_mp_exptmod_fast.c, bn_mp_mul.c, bn_mp_sqr.c, bn_mp_reduce.c, and bn_s_mp_mul_high_digs.c.

A lot of work? Yes. Advantage of getting rid of the old system: using cmake to generate all the makefiles including MSVC project files. It is easy to use cmake to exclude/include files from/to the build and it can be automated.

Cmake because it is more portable than autoconf and not everyone hates it. It also has a GUI.

The use of Cmake would also make this PR obsolete ;-)

Until than: this PR can be extended to include a (G)UI for ticking off boxes (including the small one-line explanations in the individual files as help) which I personally wouldn't find very useful but, as you ~~already gave away~~ rightfully remarked, it can also be extended without much effort to work as a (simple) parser for the user's own code to allow for a tailored version of LTM which is quite useful.
That list can replace the content of tommath_class.h and would make tommath_superclass.h obsolete. Changing the makefiles, too, to get rid of the "empty shells" would be optional.

Problems are with bn_deprecated.c which is, well, deprecated so it doesn't matter much, bn_conversion.c that should get splitted into individual files, and…

This should be reflected via the configuration such that mp_mul does not automatically require karatsuba.

You can only do so much in a q&d hack in the middle of the night ;-)

That would be the work for an option like --small-code or something in that direction.

sjaeckel · 2019-05-29T12:00:36Z

i.e., the weak dependencies.

I don't know what you mean by weak dependencies? the specific implementations? e.g. TM-mul, Kara-mul etc. for mul?!? so you want a whitelist of functions?

Cmake because it is more portable than autoconf and not everyone hates it. It also has a GUI.

@czurnieden said Cmake... I say ccmake (for the ncurses UI)

and I'm not sure what @minad will say... hooray.. or nay

minad · 2019-05-29T12:04:14Z

Please no cmake and please no autotools.

I don't know what you mean by weak dependencies? the specific implementations? e.g. TM-mul, Kara-mul etc. for mul?!? so you want a whitelist of functions?

mp_mul optionally takes advantage of s_mp_mul_karatsuba but it does not necessarily depend on it since there is a preprocessor ifdef. This I call a "weak dependency" (or optional dependency). In contrast to that, mp_add necessarily depends on s_mp_add. This is a (strong) dependency. In tommath_class.h the BN_MP_MUL_C block defines BN_S_MP_MUL_KARATSUBA_C since it has no concept of a weak dependency.

minad · 2019-05-29T12:07:34Z

What I want:

If I enable mp_mul - s_mp_mul_karatsuba should not be enabled automatically too. I want to enable that one manually since it is a weak dependency. However strong dependencies should be enabled automatically. I want a white list + strong transitive dependencies.

sjaeckel · 2019-05-29T12:09:12Z

If I enable mp_mul - s_mp_mul_karatsuba should not be enabled automatically too. I want to enable that one manually since it is a weak dependency. However strong dependencies should be enabled automatically. I want a white list + strong transitive dependencies.

okay, if you think you can solve this problem, go on! [Edit] ...and you really think it's worth the effort...[/Edit]

as long as you didn't solve it, simply add this to tommath_superclass.h after your defines

#   ifdef LTM_LAST
#      undef BN_S_MP_KARATSUBA_MUL_C
#   endif

minad · 2019-05-29T12:12:52Z

okay, if you think you can solve this problem, go on!

Given #262 and MP_HAS it is pretty easy since this allows to heuristically detect optional dependencies. And it could be integrated easily with the current infrastructure.

sjaeckel · 2019-05-31T06:09:43Z

Given #262 and MP_HAS it is pretty easy since this allows to heuristically detect optional dependencies. And it could be integrated easily with the current infrastructure.

can you please provide an example/rough sketch on top of #262 ?

minad · 2019-05-31T07:11:58Z

etc/make_small_lib.pl

+
+    $content =~ s{/\*.*?\*/}{}gs;
+    my $list = "";
+    foreach my $line (split /\n/, $content) {


does this already work together with bn_conversion bn_deprecated?

bn_deprecated? It's deprecated, don't care.
bn_conversion? Not checked yet, but unlikely. If it gets splitted into single files it will work.

It also points out a disadvantage: this script relies on LTM's naming scheme for the source files instead of their content.
The naming scheme is needed because the content of some files get generated by a preprocessor macro. It's down to only one file now, bn_conversion. Please run gcc -E and C&P the generated code at the end into correctly named individual files. That would help a lot.

No, wait, there is also bn_cutoffs.c, which needs to be treated separately. But a single file with a handful of constants is acceptable.

minad · 2019-05-31T07:12:18Z

etc/make_small_lib.pl

+       s_mp_toom_mul               => 1,
+       s_mp_toom_sqr               => 1
+   );
+my @dependency_list = ();


this is for weak dependencies I assume?

(Note to self: be a bit more verbose in commenting the code)

Yes.
But it is hardcoded and you can only switch all on/off for now.

It also ignores the extra-small alternatives to some functions as listed in the example for RSA in tommath_superclass.h

minad · 2019-05-31T07:14:50Z

etc/make_small_lib.pl

+{
+  my ($path, @entries) = @_;
+  my $tcpath;
+  my $content = "/* LibTomMath, multiple-precision integer library -- Tom St Denis */\n";


Here you are just writing a flat list right? I would suggest we remove tommath_class and tommath_superclass.h and replace it with tommath_config.h just containing the flat list. This would be preferable to having multiple different mechanisms around.

Here you are just writing a flat list right?

Yes?

I would suggest we remove tommath_class and tommath_superclass.h and replace it with tommath_config.h just containing the flat list.

There might be users who already have some kind of automatisms installed. I would like to avoid such brutality just for the sake of prettiness.

There might be users who already have some kind of automatisms installed.

IMO that's the way we should go. Why not create a fresh tommath_superclass.h from this tool?

minad · 2019-05-31T07:15:18Z

etc/make_small_lib.pl

+  open my $fh, "<", $f or die "FATAL: read_rawfile() cannot open file '$f': $!";
+  binmode $fh;
+  return do { local $/; <$fh> };
+}


File::Slurper?

To quote the aptly named Cenk K. Uygur of TyT fame: "Have at it, Hoss!" ;-)

This file contains a lot of C&P from helper.pl which has been glued together with nothing more but vague memories from bygone times, so don't expect too much, thank you.

minad · 2019-05-31T07:15:49Z

etc/make_small_lib.pl

+  print $fh $data or die "FATAL: write_file() cannot write to '$f': $!";
+  close $fh or die "FATAL: write_file() cannot close '$f': $!";
+  return;
+}


File::Slurper...come on, it should be used because of the name already

While we are at it: this one is harmless but others are not. You might look up all pronounceable names at urbandictionary.com first. (Example: slong for signed long).

?

There was a regime change here at Github, forgot?
This includes a now significant chance that somebody (e.g.: one of the shareholders) complains and another somebody (e.g.: an intern at MS) runs a badly written script over the sources here.
I don't think that it will get as bad as the rotten mess that currently happens over at YT but I also don't want to learn the differences between github and e.g.: gitlab.

But serious: no unintended(!) giggles, please, thank you.

minad · 2019-05-31T07:17:39Z

etc/make_small_lib.pl

+my $change_makefiles = 0;
+
+GetOptions( "s|source-dir=s"        => \$source_dir,
+            "t|tommath-dir=s"       => \$tommath_dir,


Could we avoid the over configuration? Just always run in ./

Just always run in ./

That's the default and it was cheap to offer options.

You can also install it and use if for different versions of LTM. Not useful now but once a new version comes out where all the deprecated stuff is actually gone you might need to keep older versions for legacy reasons.
And don't ignore the legacy part, there are people out there who make a nice penny from supporting old software and even bought an old laptop from ebay to get a fully functional parport to be able to read the backups stored with zipdrives ;-)

The goal of this script has changed a little bit. It is now a freestanding tool to adapt LTM to the user's needs only, like @sjaeckel suggested. It is also a bit more "Unix-y" this way.

The goal of this script has changed a little bit. It is now a freestanding tool to adapt LTM to the user's needs only, like @sjaeckel suggested. It is also a bit more "Unix-y" this way.

Why do we need this as a freestanding tool? I don't get that. Did @sjaeckel order this? ;)

Why do we need this as a freestanding tool?

To part the tools for the LTM developers from the tools for the users of LTM. The latter is gathered in etc for now and the first one in helper.pl.
It is not the best way to do it but the organisation in LTM is a bit odd. The documentation is in doc, the logs are in logs, in mtest is the stuff necessary for that test and a copy of Michael Fromberger's mpi.c, in demo is the testing stuff but no demo, and etc is quite a mixed bag. All in all a real mess. The organization scheme common in many larger source-trees src, build, docs, scripts, and tests with the configuration stuff in the root directory is not that bad and can be used for LTM, too.

But that's a lot of work with no gain but that satisfied feeling once it's done and until than…

Did @sjaeckel order this?

I'm no-one's servant nor any-one's master; will neither give nor take orders.

Unless you pay me well, of course, such that I can pay a professional to look over my english grammar and point out all the ambiguous parts in my texts, too.

Why do we need this as a freestanding tool?

I'm seeing this as the prototype-phase w/o interfering with helper.pl too much (less merge conflicts and stuff) so it's fine like that

Did @sjaeckel order this? ;)

lol

no but I would've started like that as well I guess ;)

@czurnieden Just to clarify, this was a joke. Sorry if you got offended. Or probably I got caught by you joking back...

@sjaeckel I agree fully to have it freestanding for the prototyping phase. But above, @czurnieden said

The goal of this script has changed a little bit. It is now a freestanding tool to adapt LTM to the user's needs only

Just to clarify, this was a joke.

You put a smiley at the end, so…?

Sorry if you got offended

That's nigh impossible, that bar hangs very high, it needs a lot to make this friendly bear remembering that a bear is omnivore.
And also: as somebody who dishes out with both hands I can hardly complain if I have to take some, too ;-)

Or probably I got caught by you joking back...

Yeah, I'm too complicated sometimes. The play with the increasingly limping metre in "I'm no-one's servant…" was probably a bit over the top?

But back to biz:

@sjaeckel I agree fully to have it freestanding for the prototyping phase. But above, @czurnieden said "The goal of this script has changed a little bit. It is now a freestanding tool to adapt LTM to the user's needs only"

Yes, we have different points of view. I don't see it as a bad thing. especially if the outcome is the same ;-)

I named the functions I copied from helper.pl or gen.pl differently if I changed them and kept their names otherwise, as you would expect. With the exception of some clashes in the short options it should be no problem to put them all in helper.pl in about half an hour, including the essential coffee break. It would be a much bigger effort the other way around.

Or to put it more bluntly: insisting on putting it all into helper.pl now comes too early.

minad · 2019-05-31T07:19:48Z

etc/make_small_lib.pl

+  }
+  else {
+    write_header($td, @dependency_list);
+  }


I would prefer if there is a single --update commandline, and then both makefiles and tommath_config.h header are rewritten.

Furthermore I would love to have an amalgam being generated too, but I could also contribute this at some later point 😀 It would replace gen.pl

EDIT: Since you still have the depmap, maybe you could also import the callgraph logic from helper.pl? While we don't commit the callgraph anymore in git (which is good), I like it to debug the dependency scanning.

I would prefer if there is a single --update commandline, and then both makefiles and tommath_config.h header are rewritten.

You need only one to make it work, but is is of course no problem to remove this condition and allow both options simultaneously.

Furthermore I would love to have an amalgam being generated too

Give them an inch and they'll take an ell. ;-)

I'd rather change gen.pl to read from tommath_class.h which has either everything listed between LTM_ALL and endif or contains the reduced set generated by this script. It would also be more "Unix-y".
That would ignore any manually made changes in tommath_superclass.h, of course, but the current gen.pl ignores them, too.

Since you still have the depmap, maybe you could also import the callgraph logic from helper.pl?

In one comment you ask: "Could we avoid the over configuration?" and here you want something extra? ;-)

The callgraph logic from helper.pls is already used in this script, didn't want to reinvent the wheel.

minad · 2019-05-31T07:27:49Z

can you please provide an example/rough sketch on top of #262 ?

Yes, the question is if I should base it on the current config infrastructure or on the script @czurnieden is working on. It already looks quite good what he is doing here I think. However I am not sure if it already handles bn_deprecated and bn_conversion correctly.

In the meantime the PRs #292, #300 and #304 are ready from my pov if you don't want the backlog to grow too long. EDIT: Also #303 if you agree with removing the separate function.

czurnieden · 2019-06-03T13:44:04Z

Note to self: be careful with git rebase --skip because it doesn't work the way you think it works and will mess everything up in a way that the normal commands for roll-back do not work as expected either and make an even bigger mess.
*sigh*

czurnieden · 2019-06-04T13:17:07Z

The script etc/make_small_lib.pl mirrors the functions oftommath_class.h/tommath_superclass.h now, plus the abilty to parse a given file/directory for LTM-functions to make a taylored LTM-library.

The file put_all_in_one_file.pl in the root directory replaces gen.pl and has the same functionality if called without arguments (The occurrence in makefile has been renamed). It has the additional function to change the path and name of the output file, use a configuration file (it can use tommath_class.h generated by etc/make_small_lib.pl), and to make the functions in the result static.

I do not plan to add more features but I might have forgotten some, so feel free to shout at me for doing so.

All of the functions in bn_conversion.c have been put into single files. It could have been automated by the script but with all of the different compilers out there and their different ways to run the preprocessor stage only (if they are able to do so at all) it would need to do the preprocessor stage itself and strip the functions out of it and place them in a file and introduce that file to the build-process and whatever I have forgotten to do, too.
Uhm, sorry, but: no.
But I won't complain if you want do do it, of course ;-)

The splitting of bn_conversion.c could have been a separate PR but I think it is just formatting, not worth one. (Yes, I keep up with @fperrad 's lintings and all other changes).

Some folks here want to weave these scripts into helper.pl but I think helper.pl is already overloaded with functionality and shouldn't have more options than ls.
But I'm open to good arguments, of course.

And now let me see how many bugs my product actually has. *sigh*

…_class.h if generated by etc/make_small_lib.pl

minad · 2019-06-07T02:34:04Z

Regarding having this as a separate tool: I think it would be better to have this as part of helper.pl since we are doing all the generation tasks there right now. Similar to calltree.txt, tge statified amalgam could be .gitignored and generated always for helper.pl --update-files. I like what you are doing here generally. I would just like it a bit simpler with less config and less moving parts. Only config file + helper.pl - gen.pl.

sjaeckel · 2019-06-07T06:54:56Z

Just an idea

What do you think about having a tool which can be dropped in like $CC and writes out a list of potential ltm-functioncalls? So I can take my project and do something like

$ CC=../libtommath/tool.py make
$ ../libtommath/tool.py -l ltm_list.txt > ../libtommath/tommath_superclass.h

and probably @fperrad or @karel-m who have AFAICT the biggest Perl skills here have something to add as well (probably some ideas, comments or even some sneaky implementation ;) )

minad · 2019-06-07T08:58:20Z

Just an idea
What do you think about having a tool which can be dropped in like...

You mentioned this idea of a scanner for external projects before. For me it would be of limited usefulness in contrast to what @czurnieden proposes here.

I am using such a minimal ltm subset, but right now I simply select the needed functions by hand. Given the config system proposed here, my configuration process that would be even simpler. But usually I only have to do this once. An external scanner wouldn't be used often, since the scanner would only touch the config (tommath_superclass.h or the config file introduced in this PR).

However I would suggest we discuss the external scanner in a separate issue, since it is independent of this PR. It can be made such that it works with the current config system and it could be made such that it works with the new system proposed by czurnieden here.

czurnieden · 2019-06-07T12:45:17Z

Just an idea

(Scary! And in bold even! ;-) )

What do you think about having a tool which can be dropped in like $CC and writes out a list of potential ltm-functioncalls?

It's currently implemented in etc/make_small_lib.pl as

libtommath $ ./etc/make_small_lib.pl --source-dir=/path/to/your/sources --tommath-dir=../
libtommath $ make clean && make
libtommath $ cd /path/to/your/sources
sources $ LIBTOMMATH="../../../libtommath/libtommath.a" make

The default is to write a new tommath_class.h and empty tommath_superclass.h (an option to build the old RSA example and to skip optimizations is also included), so if you need a written list, it is in tommath_class.h (as defines, so you would need a round of sed if you want to do more with it).

It has the option to manipulate the makefiles to scrape another couple of bytes off by not producing "empty shells" (yes, I am aware of e.g.: ar -d member).

This script does not make backups of the manipulated files yet, it relies on being run in an active git repository and its ability to do a git checkout -- tommath* make*.

a bit simpler with less config and less moving parts.

There is no distinct configuration file needed anymore, etc/make_small_lib.pl writes to tommath_class.h which you can feed to put_all_in_one_file.pl (backwards compatible to gen.pl) without further ado.
With the example from above it is just one extra step:

libtommath $ ./etc/make_small_lib.pl --source-dir=/path/to/your/sources --tommath-dir=../
libtommath $ ./put_all_in_one_file.pl --config-file=tommath_class.h --output=/path/to/your/sources/allinone.c --make-static

tommath.h doesn't get any massage yet, so if you need a kind of tommath_small.h you need to do it yourself for now.

Functionality of --make-static depends on the outcome of #313 and might get dropped if it's too much hassle (The scripts depend on static, individual files, correctly named. A need to fiddle with the preprocessor is a good example for "too much hassle" ).

However I would suggest we discuss the external scanner in a separate issue, since it is independent of this PR.

It's already build in, it was a by-product.

@fperrad or @karel-m who have AFAICT the biggest Perl skills here have something to add as well

That would be highly appreciated!

minad · 2020-02-20T22:08:19Z

Close, see #160 for the reason

czurnieden mentioned this pull request May 28, 2019

Alternative algorithm in mp_n_root_ex #202

Closed

minad reviewed May 29, 2019

View reviewed changes

make_config.pl Outdated Show resolved Hide resolved

minad mentioned this pull request May 29, 2019

Better interoperability between MSVC and mingw-w64, 3 #300

Merged

czurnieden added the work in progress label May 31, 2019

czurnieden force-pushed the make_config branch from 033cf41 to e323adc Compare May 31, 2019 01:53

minad reviewed May 31, 2019

View reviewed changes

czurnieden force-pushed the make_config branch 2 times, most recently from e1b0470 to 681925a Compare May 31, 2019 23:30

czurnieden force-pushed the make_config branch 3 times, most recently from 278f50f to 961b277 Compare June 3, 2019 13:25

sjaeckel mentioned this pull request Jun 5, 2019

libtomath build broken on win32, due to the use of uint8_t #104

Closed

This was referenced Jun 5, 2019

Removed bn_conversion.c in favor of individual files #311

Closed

The new approach used in bn_conversion.c makes it impossible to keep no-stdint-h #309

Closed

czurnieden added 10 commits June 6, 2019 13:15

First sketch for an easier configuration

439cf72

extended functionality and moved it to libtommath/etc

b5fb086

allow for both, changing headers and makfiles

abd593a

checking for deprecated functions

592c964

inclusion of mp_(get|set)_(u|i)(32|64)

370a15b

replaced gen.pl with put_all_in_one_file.pl that is able read tommath…

d501892

…_class.h if generated by etc/make_small_lib.pl

replaced DIY 'uniq' with the one from List::MoreUtils

1747247

Allow use of deprecated functions, too

1d6f12d

additional options to build RSA packets

bc6c0b2

tbs

0e602d2

czurnieden force-pushed the make_config branch from cc8fc5a to 0e602d2 Compare June 6, 2019 11:16

minad mentioned this pull request Jun 6, 2019

#311 with macros #313

Merged

minad mentioned this pull request Oct 17, 2019

Addition of fast division (recursive divrem only) #370

Merged

czurnieden added this to the v2.0.0 milestone Oct 17, 2019

This was referenced Oct 27, 2019

functions guarded by MP_HAS are treated as optional dependencies #424

Closed

move to c99 #416

Closed

minad closed this Feb 20, 2020

First sketch for an easier configuration #301

First sketch for an easier configuration #301

Conversation

czurnieden commented May 28, 2019

minad commented May 29, 2019

sjaeckel commented May 29, 2019 • edited Loading

sjaeckel commented May 29, 2019

minad commented May 29, 2019

sjaeckel commented May 29, 2019

minad commented May 29, 2019

sjaeckel commented May 29, 2019

minad commented May 29, 2019

czurnieden commented May 29, 2019

sjaeckel commented May 29, 2019

minad commented May 29, 2019

minad commented May 29, 2019

sjaeckel commented May 29, 2019 • edited Loading

minad commented May 29, 2019

sjaeckel commented May 31, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

minad Jun 7, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

minad May 31, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

minad commented May 31, 2019 • edited Loading

czurnieden commented Jun 3, 2019

czurnieden commented Jun 4, 2019

minad commented Jun 7, 2019

sjaeckel commented Jun 7, 2019

minad commented Jun 7, 2019

czurnieden commented Jun 7, 2019

minad commented Feb 20, 2020

sjaeckel commented May 29, 2019 •

edited

Loading

sjaeckel commented May 29, 2019 •

edited

Loading

minad Jun 7, 2019 •

edited

Loading

minad May 31, 2019 •

edited

Loading

minad commented May 31, 2019 •

edited

Loading