Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set SQLITE_LIMIT_FUNCTION_ARG to SQLite's normal default. #2493

Merged
merged 1 commit into from
Aug 9, 2024

Conversation

kentonv
Copy link
Member

@kentonv kentonv commented Aug 7, 2024

People keep legitimately running up against this limit. It's not really clear what value it is providing, so let's just match the default.

Fixes #2412

People keep legitimately running up against this limit. It's not really clear what value it is providing, so let's just match the default.

Fixes #2412
@kentonv kentonv requested review from elithrar and geelen August 7, 2024 18:23
@kentonv kentonv requested review from a team as code owners August 7, 2024 18:23
@justin-mp
Copy link
Contributor

Just to add color here, according to https://www.sqlite.org/limits.html (search for "Maximum Number Of Arguments On A Function"), the default is 100 and absolute maximum is 127 because this value is sometimes stored in an unsigned char.

Sure enough, sqliteLimit.h has:

#ifndef SQLITE_MAX_FUNCTION_ARG
# define SQLITE_MAX_FUNCTION_ARG 127
#endif

I can't seem to figure out where the default of 100 is set.

@@ -809,7 +809,9 @@ void SqliteDatabase::setupSecurity() {
// https://www.sqlite.org/limits.html#max_compound_select
sqlite3_limit(db, SQLITE_LIMIT_COMPOUND_SELECT, 5);
sqlite3_limit(db, SQLITE_LIMIT_VDBE_OP, 25000);
sqlite3_limit(db, SQLITE_LIMIT_FUNCTION_ARG, 32);
// For SQLITE_LIMIT_FUNCTION_ARG we use the default instead of the "security" recommendation
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: s/default/maximum/ ?

@kentonv
Copy link
Member Author

kentonv commented Aug 7, 2024

Weird, https://www.sqlite.org/security.html says that the default is 127, not 100.

My intent was to use the default.

I guess I can try to find out what sqlite actually reports its own default as being...

@justin-mp
Copy link
Contributor

justin-mp commented Aug 7, 2024

Digging more: the default is 127 and https://www.sqlite.org/limits.html is buggy. Here's what my sqlite3 reports:

sqlite> .limit
              length 1000000000
          sql_length 1000000000
              column 2000
          expr_depth 1000
     compound_select 500
             vdbe_op 250000000
        function_arg 127
            attached 10
 like_pattern_length 50000
     variable_number 250000
       trigger_depth 1000
      worker_threads 0

The limits from the compile-time constants are initialized into a static const int aHardLimit[] and then that array is copied into the per-database limit in openDatabase(...):

  assert( sizeof(db->aLimit)==sizeof(aHardLimit) );
  memcpy(db->aLimit, aHardLimit, sizeof(db->aLimit));

So it's the default and maximum.

sqlite3_limit(db, SQLITE_LIMIT_FUNCTION_ARG, 32);
// For SQLITE_LIMIT_FUNCTION_ARG we use the default instead of the "security" recommendation
// because there are too many valid use cases for large argument lists, especially json_object.
sqlite3_limit(db, SQLITE_LIMIT_FUNCTION_ARG, 127);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this might change in the future, maybe we should add a link to the existing documentation?

@anonrig
Copy link
Member

anonrig commented Aug 8, 2024

Weird, https://www.sqlite.org/security.html says that the default is 127, not 100.

PS: https://www.sqlite.org/c3ref/create_function.html Also mentions max 127 value.

@kentonv
Copy link
Member Author

kentonv commented Aug 9, 2024

OK seems like this is correct as-is, just gonna merge.

@kentonv kentonv merged commit e0c1e78 into main Aug 9, 2024
9 checks passed
@kentonv kentonv deleted the kenton/sqlite-arg-limit branch August 9, 2024 21:27
@HendrixString
Copy link

@kentonv Hi,

when is this gonna be released up stream into production ?

Unfortunately, i am hitting the upper limit of 32 (I have 36 params in json_object)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Error: too many arguments on function JSON_OBJECT - 🐛 Bug Report — Runtime APIs
4 participants