Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
Comment: | Update the built-in SQLite to the latest trunk version that includes the order-by-subquery optimization. |
---|---|
Downloads: | Tarball | ZIP archive |
Timelines: | family | ancestors | descendants | both | trunk |
Files: | files | file ages | folders |
SHA3-256: |
a8aaed421614410c34e1bf9cddd44693 |
User & Date: | drh 2024-08-16 19:06:24.772 |
Context
2024-08-16
| ||
19:24 | Fix the query in the compute_youngest_ancestor_in_branch() routine so that it has an ORDER BY to make it safe. The new order-by-subquery optimization in SQLite turns that ORDER BY into a no-op, so there is no performance impact. ... (check-in: 60debc7e user: drh tags: trunk) | |
19:06 | Update the built-in SQLite to the latest trunk version that includes the order-by-subquery optimization. ... (check-in: a8aaed42 user: drh tags: trunk) | |
2024-08-15
| ||
14:11 | Add a /robots.txt page - useful only when Fossil is deployed as a stand-alone server instead of as a sub-component to a larger website. ... (check-in: cadfcba3 user: drh tags: trunk) | |
Changes
Changes to extsrc/shell.c.
︙ | ︙ | |||
2844 2845 2846 2847 2848 2849 2850 | ****************************************************************************** ** ** This SQLite extension implements functions that compute SHA3 hashes ** in the way described by the (U.S.) NIST FIPS 202 SHA-3 Standard. ** Two SQL functions are implemented: ** ** sha3(X,SIZE) | > | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 2844 2845 2846 2847 2848 2849 2850 2851 2852 2853 2854 2855 2856 2857 2858 2859 2860 2861 2862 2863 2864 2865 2866 2867 2868 2869 2870 2871 2872 2873 2874 2875 2876 2877 2878 2879 2880 2881 2882 2883 2884 2885 2886 2887 2888 2889 2890 2891 2892 2893 2894 2895 2896 2897 2898 2899 2900 2901 2902 2903 2904 2905 2906 2907 2908 2909 2910 2911 2912 2913 2914 2915 2916 2917 2918 2919 2920 2921 2922 2923 2924 2925 2926 2927 2928 2929 2930 2931 2932 2933 2934 2935 2936 2937 2938 2939 2940 | ****************************************************************************** ** ** This SQLite extension implements functions that compute SHA3 hashes ** in the way described by the (U.S.) NIST FIPS 202 SHA-3 Standard. ** Two SQL functions are implemented: ** ** sha3(X,SIZE) ** sha3_agg(Y,SIZE) ** sha3_query(Z,SIZE) ** ** The sha3(X) function computes the SHA3 hash of the input X, or NULL if ** X is NULL. If inputs X is text, the UTF-8 rendering of that text is ** used to compute the hash. If X is a BLOB, then the binary data of the ** blob is used to compute the hash. If X is an integer or real number, ** then that number if converted into UTF-8 text and the hash is computed ** over the text. ** ** The sha3_agg(Y) function computes the SHA3 hash of all Y inputs. Since ** order is important for the hash, it is recommended that the Y expression ** by followed by an ORDER BY clause to guarantee that the inputs occur ** in the desired order. ** ** The sha3_query(Y) function evaluates all queries in the SQL statements of Y ** and returns a hash of their results. ** ** The SIZE argument is optional. If omitted, the SHA3-256 hash algorithm ** is used. If SIZE is included it must be one of the integers 224, 256, ** 384, or 512, to determine SHA3 hash variant that is computed. ** ** Because the sha3_agg() and sha3_query() functions compute a hash over ** multiple values, the values are encode to use include type information. ** ** In sha3_agg(), the sequence of bytes that gets hashed for each input ** Y depends on the datatype of Y: ** ** typeof(Y)='null' A single "N" is hashed. (One byte) ** ** typeof(Y)='integer' The data hash is the character "I" followed ** by an 8-byte big-endian binary of the ** 64-bit signed integer. (Nine bytes total.) ** ** typeof(Y)='real' The character "F" followed by an 8-byte ** big-ending binary of the double. (Nine ** bytes total.) ** ** typeof(Y)='text' The hash is over prefix "Tnnn:" followed ** by the UTF8 encoding of the text. The "nnn" ** in the prefix is the minimum-length decimal ** representation of the octet_length of the text. ** Notice the ":" at the end of the prefix, which ** is needed to separate the prefix from the ** content in cases where the content starts ** with a digit. ** ** typeof(Y)='blob' The hash is taken over prefix "Bnnn:" followed ** by the binary content of the blob. The "nnn" ** in the prefix is the mimimum-length decimal ** representation of the byte-length of the blob. ** ** According to the rules above, all of the following SELECT statements ** should return TRUE: ** ** SELECT sha3(1) = sha3('1'); ** ** SELECT sha3('hello') = sha3(x'68656c6c6f'); ** ** WITH a(x) AS (VALUES('xyzzy')) ** SELECT sha3_agg(x) = sha3('T5:xyzzy') FROM a; ** ** WITH a(x) AS (VALUES(x'010203')) ** SELECT sha3_agg(x) = sha3(x'42333a010203') FROM a; ** ** WITH a(x) AS (VALUES(0x123456)) ** SELECT sha3_agg(x) = sha3(x'490000000000123456') FROM a; ** ** WITH a(x) AS (VALUES(100.015625)) ** SELECT sha3_agg(x) = sha3(x'464059010000000000') FROM a; ** ** WITH a(x) AS (VALUES(NULL)) ** SELECT sha3_agg(x) = sha3('N') FROM a; ** ** ** In sha3_query(), individual column values are encoded as with ** sha3_agg(), but with the addition that a single "R" character is ** inserted at the start of each row. ** ** Note that sha3_agg() hashes rows for which Y is NULL. Add a FILTER ** clause if NULL rows should be excluded: ** ** SELECT sha3_agg(x ORDER BY rowid) FILTER(WHERE x NOT NULL) FROM t1; */ /* #include "sqlite3ext.h" */ SQLITE_EXTENSION_INIT1 #include <assert.h> #include <string.h> #include <stdarg.h> |
︙ | ︙ | |||
2904 2905 2906 2907 2908 2909 2910 2911 2912 2913 2914 2915 2916 2917 | union { u64 s[25]; /* Keccak state. 5x5 lines of 64 bits each */ unsigned char x[1600]; /* ... or 1600 bytes */ } u; unsigned nRate; /* Bytes of input accepted per Keccak iteration */ unsigned nLoaded; /* Input bytes loaded into u.x[] so far this cycle */ unsigned ixMask; /* Insert next input into u.x[nLoaded^ixMask]. */ }; /* ** A single step of the Keccak mixing function for a 1600-bit state */ static void KeccakF1600Step(SHA3Context *p){ int i; | > | 2976 2977 2978 2979 2980 2981 2982 2983 2984 2985 2986 2987 2988 2989 2990 | union { u64 s[25]; /* Keccak state. 5x5 lines of 64 bits each */ unsigned char x[1600]; /* ... or 1600 bytes */ } u; unsigned nRate; /* Bytes of input accepted per Keccak iteration */ unsigned nLoaded; /* Input bytes loaded into u.x[] so far this cycle */ unsigned ixMask; /* Insert next input into u.x[nLoaded^ixMask]. */ unsigned iSize; /* 224, 256, 358, or 512 */ }; /* ** A single step of the Keccak mixing function for a 1600-bit state */ static void KeccakF1600Step(SHA3Context *p){ int i; |
︙ | ︙ | |||
3233 3234 3235 3236 3237 3238 3239 3240 3241 3242 3243 3244 3245 3246 | /* ** Initialize a new hash. iSize determines the size of the hash ** in bits and should be one of 224, 256, 384, or 512. Or iSize ** can be zero to use the default hash size of 256 bits. */ static void SHA3Init(SHA3Context *p, int iSize){ memset(p, 0, sizeof(*p)); if( iSize>=128 && iSize<=512 ){ p->nRate = (1600 - ((iSize + 31)&~31)*2)/8; }else{ p->nRate = (1600 - 2*256)/8; } #if SHA3_BYTEORDER==1234 /* Known to be little-endian at compile-time. No-op */ | > | 3306 3307 3308 3309 3310 3311 3312 3313 3314 3315 3316 3317 3318 3319 3320 | /* ** Initialize a new hash. iSize determines the size of the hash ** in bits and should be one of 224, 256, 384, or 512. Or iSize ** can be zero to use the default hash size of 256 bits. */ static void SHA3Init(SHA3Context *p, int iSize){ memset(p, 0, sizeof(*p)); p->iSize = iSize; if( iSize>=128 && iSize<=512 ){ p->nRate = (1600 - ((iSize + 31)&~31)*2)/8; }else{ p->nRate = (1600 - 2*256)/8; } #if SHA3_BYTEORDER==1234 /* Known to be little-endian at compile-time. No-op */ |
︙ | ︙ | |||
3375 3376 3377 3378 3379 3380 3381 3382 3383 3384 3385 3386 3387 3388 | char zBuf[50]; va_start(ap, zFormat); sqlite3_vsnprintf(sizeof(zBuf),zBuf,zFormat,ap); va_end(ap); n = (int)strlen(zBuf); SHA3Update(p, (unsigned char*)zBuf, n); } /* ** Implementation of the sha3_query(SQL,SIZE) function. ** ** This function compiles and runs the SQL statement(s) given in the ** argument. The results are hashed using a SIZE-bit SHA3. The default ** size is 256. | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 3449 3450 3451 3452 3453 3454 3455 3456 3457 3458 3459 3460 3461 3462 3463 3464 3465 3466 3467 3468 3469 3470 3471 3472 3473 3474 3475 3476 3477 3478 3479 3480 3481 3482 3483 3484 3485 3486 3487 3488 3489 3490 3491 3492 3493 3494 3495 3496 3497 3498 3499 3500 3501 3502 3503 3504 3505 3506 3507 3508 3509 3510 3511 3512 3513 3514 3515 3516 | char zBuf[50]; va_start(ap, zFormat); sqlite3_vsnprintf(sizeof(zBuf),zBuf,zFormat,ap); va_end(ap); n = (int)strlen(zBuf); SHA3Update(p, (unsigned char*)zBuf, n); } /* ** Update a SHA3Context using a single sqlite3_value. */ static void sha3UpdateFromValue(SHA3Context *p, sqlite3_value *pVal){ switch( sqlite3_value_type(pVal) ){ case SQLITE_NULL: { SHA3Update(p, (const unsigned char*)"N",1); break; } case SQLITE_INTEGER: { sqlite3_uint64 u; int j; unsigned char x[9]; sqlite3_int64 v = sqlite3_value_int64(pVal); memcpy(&u, &v, 8); for(j=8; j>=1; j--){ x[j] = u & 0xff; u >>= 8; } x[0] = 'I'; SHA3Update(p, x, 9); break; } case SQLITE_FLOAT: { sqlite3_uint64 u; int j; unsigned char x[9]; double r = sqlite3_value_double(pVal); memcpy(&u, &r, 8); for(j=8; j>=1; j--){ x[j] = u & 0xff; u >>= 8; } x[0] = 'F'; SHA3Update(p,x,9); break; } case SQLITE_TEXT: { int n2 = sqlite3_value_bytes(pVal); const unsigned char *z2 = sqlite3_value_text(pVal); sha3_step_vformat(p,"T%d:",n2); SHA3Update(p, z2, n2); break; } case SQLITE_BLOB: { int n2 = sqlite3_value_bytes(pVal); const unsigned char *z2 = sqlite3_value_blob(pVal); sha3_step_vformat(p,"B%d:",n2); SHA3Update(p, z2, n2); break; } } } /* ** Implementation of the sha3_query(SQL,SIZE) function. ** ** This function compiles and runs the SQL statement(s) given in the ** argument. The results are hashed using a SIZE-bit SHA3. The default ** size is 256. |
︙ | ︙ | |||
3465 3466 3467 3468 3469 3470 3471 | SHA3Update(&cx,(unsigned char*)z,n); } /* Compute a hash over the result of the query */ while( SQLITE_ROW==sqlite3_step(pStmt) ){ SHA3Update(&cx,(const unsigned char*)"R",1); for(i=0; i<nCol; i++){ | | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 3593 3594 3595 3596 3597 3598 3599 3600 3601 3602 3603 3604 3605 3606 3607 3608 3609 3610 3611 3612 3613 3614 3615 3616 3617 3618 3619 3620 3621 3622 3623 3624 3625 3626 3627 3628 3629 3630 3631 3632 3633 3634 3635 3636 3637 3638 3639 3640 3641 3642 3643 3644 3645 3646 3647 3648 3649 3650 3651 | SHA3Update(&cx,(unsigned char*)z,n); } /* Compute a hash over the result of the query */ while( SQLITE_ROW==sqlite3_step(pStmt) ){ SHA3Update(&cx,(const unsigned char*)"R",1); for(i=0; i<nCol; i++){ sha3UpdateFromValue(&cx, sqlite3_column_value(pStmt,i)); } } sqlite3_finalize(pStmt); } sqlite3_result_blob(context, SHA3Final(&cx), iSize/8, SQLITE_TRANSIENT); } /* ** xStep function for sha3_agg(). */ static void sha3AggStep( sqlite3_context *context, int argc, sqlite3_value **argv ){ SHA3Context *p; p = (SHA3Context*)sqlite3_aggregate_context(context, sizeof(*p)); if( p==0 ) return; if( p->nRate==0 ){ int sz = 256; if( argc==2 ){ sz = sqlite3_value_int(argv[1]); if( sz!=224 && sz!=384 && sz!=512 ){ sz = 256; } } SHA3Init(p, sz); } sha3UpdateFromValue(p, argv[0]); } /* ** xFinal function for sha3_agg(). */ static void sha3AggFinal(sqlite3_context *context){ SHA3Context *p; p = (SHA3Context*)sqlite3_aggregate_context(context, sizeof(*p)); if( p==0 ) return; if( p->iSize ){ sqlite3_result_blob(context, SHA3Final(p), p->iSize/8, SQLITE_TRANSIENT); } } #ifdef _WIN32 #endif int sqlite3_shathree_init( sqlite3 *db, |
︙ | ︙ | |||
3540 3541 3542 3543 3544 3545 3546 3547 3548 3549 3550 3551 3552 3553 | SQLITE_UTF8 | SQLITE_INNOCUOUS | SQLITE_DETERMINISTIC, 0, sha3Func, 0, 0); if( rc==SQLITE_OK ){ rc = sqlite3_create_function(db, "sha3", 2, SQLITE_UTF8 | SQLITE_INNOCUOUS | SQLITE_DETERMINISTIC, 0, sha3Func, 0, 0); } if( rc==SQLITE_OK ){ rc = sqlite3_create_function(db, "sha3_query", 1, SQLITE_UTF8 | SQLITE_DIRECTONLY, 0, sha3QueryFunc, 0, 0); } if( rc==SQLITE_OK ){ rc = sqlite3_create_function(db, "sha3_query", 2, | > > > > > > > > > > | 3659 3660 3661 3662 3663 3664 3665 3666 3667 3668 3669 3670 3671 3672 3673 3674 3675 3676 3677 3678 3679 3680 3681 3682 | SQLITE_UTF8 | SQLITE_INNOCUOUS | SQLITE_DETERMINISTIC, 0, sha3Func, 0, 0); if( rc==SQLITE_OK ){ rc = sqlite3_create_function(db, "sha3", 2, SQLITE_UTF8 | SQLITE_INNOCUOUS | SQLITE_DETERMINISTIC, 0, sha3Func, 0, 0); } if( rc==SQLITE_OK ){ rc = sqlite3_create_function(db, "sha3_agg", 1, SQLITE_UTF8 | SQLITE_INNOCUOUS | SQLITE_DETERMINISTIC, 0, 0, sha3AggStep, sha3AggFinal); } if( rc==SQLITE_OK ){ rc = sqlite3_create_function(db, "sha3_agg", 2, SQLITE_UTF8 | SQLITE_INNOCUOUS | SQLITE_DETERMINISTIC, 0, 0, sha3AggStep, sha3AggFinal); } if( rc==SQLITE_OK ){ rc = sqlite3_create_function(db, "sha3_query", 1, SQLITE_UTF8 | SQLITE_DIRECTONLY, 0, sha3QueryFunc, 0, 0); } if( rc==SQLITE_OK ){ rc = sqlite3_create_function(db, "sha3_query", 2, |
︙ | ︙ |
Changes to extsrc/sqlite3.c.
︙ | ︙ | |||
14 15 16 17 18 19 20 | ** the text of this file. Search for "Begin file sqlite3.h" to find the start ** of the embedded sqlite3.h header file.) Additional code files may be needed ** if you want a wrapper to interface SQLite with your choice of programming ** language. The code for the "sqlite3" command-line shell is also in a ** separate file. This file contains only code for the core SQLite library. ** ** The content in this amalgamation comes from Fossil check-in | | | 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | ** the text of this file. Search for "Begin file sqlite3.h" to find the start ** of the embedded sqlite3.h header file.) Additional code files may be needed ** if you want a wrapper to interface SQLite with your choice of programming ** language. The code for the "sqlite3" command-line shell is also in a ** separate file. This file contains only code for the core SQLite library. ** ** The content in this amalgamation comes from Fossil check-in ** 7a0cdc7edb704a88a77b748cd28f6e00c498. */ #define SQLITE_CORE 1 #define SQLITE_AMALGAMATION 1 #ifndef SQLITE_PRIVATE # define SQLITE_PRIVATE static #endif /************** Begin file sqliteInt.h ***************************************/ |
︙ | ︙ | |||
460 461 462 463 464 465 466 | ** ** See also: [sqlite3_libversion()], ** [sqlite3_libversion_number()], [sqlite3_sourceid()], ** [sqlite_version()] and [sqlite_source_id()]. */ #define SQLITE_VERSION "3.47.0" #define SQLITE_VERSION_NUMBER 3047000 | | | 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 | ** ** See also: [sqlite3_libversion()], ** [sqlite3_libversion_number()], [sqlite3_sourceid()], ** [sqlite_version()] and [sqlite_source_id()]. */ #define SQLITE_VERSION "3.47.0" #define SQLITE_VERSION_NUMBER 3047000 #define SQLITE_SOURCE_ID "2024-08-16 18:51:46 7a0cdc7edb704a88a77b748cd28f6e00c49849cc2c1af838b95b34232ecc21f9" /* ** CAPI3REF: Run-Time Library Version Numbers ** KEYWORDS: sqlite3_version sqlite3_sourceid ** ** These interfaces provide the same information as the [SQLITE_VERSION], ** [SQLITE_VERSION_NUMBER], and [SQLITE_SOURCE_ID] C preprocessor macros |
︙ | ︙ | |||
17913 17914 17915 17916 17917 17918 17919 17920 17921 17922 17923 17924 17925 17926 | #define SQLITE_ReleaseReg 0x00400000 /* Use OP_ReleaseReg for testing */ #define SQLITE_FlttnUnionAll 0x00800000 /* Disable the UNION ALL flattener */ /* TH3 expects this value ^^^^^^^^^^ See flatten04.test */ #define SQLITE_IndexedExpr 0x01000000 /* Pull exprs from index when able */ #define SQLITE_Coroutines 0x02000000 /* Co-routines for subqueries */ #define SQLITE_NullUnusedCols 0x04000000 /* NULL unused columns in subqueries */ #define SQLITE_OnePass 0x08000000 /* Single-pass DELETE and UPDATE */ #define SQLITE_AllOpts 0xffffffff /* All optimizations */ /* ** Macros for testing whether or not optimizations are enabled or disabled. */ #define OptimizationDisabled(db, mask) (((db)->dbOptFlags&(mask))!=0) #define OptimizationEnabled(db, mask) (((db)->dbOptFlags&(mask))==0) | > | 17913 17914 17915 17916 17917 17918 17919 17920 17921 17922 17923 17924 17925 17926 17927 | #define SQLITE_ReleaseReg 0x00400000 /* Use OP_ReleaseReg for testing */ #define SQLITE_FlttnUnionAll 0x00800000 /* Disable the UNION ALL flattener */ /* TH3 expects this value ^^^^^^^^^^ See flatten04.test */ #define SQLITE_IndexedExpr 0x01000000 /* Pull exprs from index when able */ #define SQLITE_Coroutines 0x02000000 /* Co-routines for subqueries */ #define SQLITE_NullUnusedCols 0x04000000 /* NULL unused columns in subqueries */ #define SQLITE_OnePass 0x08000000 /* Single-pass DELETE and UPDATE */ #define SQLITE_OrderBySubq 0x10000000 /* ORDER BY in subquery helps outer */ #define SQLITE_AllOpts 0xffffffff /* All optimizations */ /* ** Macros for testing whether or not optimizations are enabled or disabled. */ #define OptimizationDisabled(db, mask) (((db)->dbOptFlags&(mask))!=0) #define OptimizationEnabled(db, mask) (((db)->dbOptFlags&(mask))==0) |
︙ | ︙ | |||
120469 120470 120471 120472 120473 120474 120475 | rc = sqlite3_prepare(db, zSql, -1, &pStmt, 0); sqlite3DbFree(db, zSql); if( rc ) return rc; while( sqlite3_step(pStmt)==SQLITE_ROW ){ int nIdxCol = 1; /* Number of columns in stat4 records */ | | | | | | | > | > > | | 120470 120471 120472 120473 120474 120475 120476 120477 120478 120479 120480 120481 120482 120483 120484 120485 120486 120487 120488 120489 120490 120491 120492 120493 120494 120495 120496 120497 120498 120499 120500 120501 120502 120503 120504 120505 120506 120507 120508 120509 120510 120511 120512 120513 120514 120515 120516 120517 120518 120519 120520 120521 | rc = sqlite3_prepare(db, zSql, -1, &pStmt, 0); sqlite3DbFree(db, zSql); if( rc ) return rc; while( sqlite3_step(pStmt)==SQLITE_ROW ){ int nIdxCol = 1; /* Number of columns in stat4 records */ char *zIndex; /* Index name */ Index *pIdx; /* Pointer to the index object */ int nSample; /* Number of samples */ int nByte; /* Bytes of space required */ int i; /* Bytes of space required */ tRowcnt *pSpace; /* Available allocated memory space */ u8 *pPtr; /* Available memory as a u8 for easier manipulation */ zIndex = (char *)sqlite3_column_text(pStmt, 0); if( zIndex==0 ) continue; nSample = sqlite3_column_int(pStmt, 1); pIdx = findIndexOrPrimaryKey(db, zIndex, zDb); assert( pIdx==0 || pIdx->nSample==0 ); if( pIdx==0 ) continue; if( pIdx->aSample!=0 ){ /* The same index appears in sqlite_stat4 under multiple names */ continue; } assert( !HasRowid(pIdx->pTable) || pIdx->nColumn==pIdx->nKeyCol+1 ); if( !HasRowid(pIdx->pTable) && IsPrimaryKeyIndex(pIdx) ){ nIdxCol = pIdx->nKeyCol; }else{ nIdxCol = pIdx->nColumn; } pIdx->nSampleCol = nIdxCol; pIdx->mxSample = nSample; nByte = ROUND8(sizeof(IndexSample) * nSample); nByte += sizeof(tRowcnt) * nIdxCol * 3 * nSample; nByte += nIdxCol * sizeof(tRowcnt); /* Space for Index.aAvgEq[] */ pIdx->aSample = sqlite3DbMallocZero(db, nByte); if( pIdx->aSample==0 ){ sqlite3_finalize(pStmt); return SQLITE_NOMEM_BKPT; } pPtr = (u8*)pIdx->aSample; pPtr += ROUND8(nSample*sizeof(pIdx->aSample[0])); pSpace = (tRowcnt*)pPtr; assert( EIGHT_BYTE_ALIGNMENT( pSpace ) ); pIdx->aAvgEq = pSpace; pSpace += nIdxCol; pIdx->pTable->tabFlags |= TF_HasStat4; for(i=0; i<nSample; i++){ pIdx->aSample[i].anEq = pSpace; pSpace += nIdxCol; pIdx->aSample[i].anLt = pSpace; pSpace += nIdxCol; pIdx->aSample[i].anDLt = pSpace; pSpace += nIdxCol; |
︙ | ︙ | |||
157186 157187 157188 157189 157190 157191 157192 157193 157194 157195 157196 157197 157198 157199 | union { struct { /* Information for internal btree tables */ u16 nEq; /* Number of equality constraints */ u16 nBtm; /* Size of BTM vector */ u16 nTop; /* Size of TOP vector */ u16 nDistinctCol; /* Index columns used to sort for DISTINCT */ Index *pIndex; /* Index used, or NULL */ } btree; struct { /* Information for virtual tables */ int idxNum; /* Index number */ u32 needFree : 1; /* True if sqlite3_free(idxStr) is needed */ u32 bOmitOffset : 1; /* True to let virtual table handle offset */ i8 isOrdered; /* True if satisfies ORDER BY */ u16 omitMask; /* Terms that may be omitted */ | > | 157190 157191 157192 157193 157194 157195 157196 157197 157198 157199 157200 157201 157202 157203 157204 | union { struct { /* Information for internal btree tables */ u16 nEq; /* Number of equality constraints */ u16 nBtm; /* Size of BTM vector */ u16 nTop; /* Size of TOP vector */ u16 nDistinctCol; /* Index columns used to sort for DISTINCT */ Index *pIndex; /* Index used, or NULL */ ExprList *pOrderBy; /* ORDER BY clause if this is really a subquery */ } btree; struct { /* Information for virtual tables */ int idxNum; /* Index number */ u32 needFree : 1; /* True if sqlite3_free(idxStr) is needed */ u32 bOmitOffset : 1; /* True to let virtual table handle offset */ i8 isOrdered; /* True if satisfies ORDER BY */ u16 omitMask; /* Terms that may be omitted */ |
︙ | ︙ | |||
157679 157680 157681 157682 157683 157684 157685 | #define WHERE_IN_EARLYOUT 0x00040000 /* Perhaps quit IN loops early */ #define WHERE_BIGNULL_SORT 0x00080000 /* Column nEq of index is BIGNULL */ #define WHERE_IN_SEEKSCAN 0x00100000 /* Seek-scan optimization for IN */ #define WHERE_TRANSCONS 0x00200000 /* Uses a transitive constraint */ #define WHERE_BLOOMFILTER 0x00400000 /* Consider using a Bloom-filter */ #define WHERE_SELFCULL 0x00800000 /* nOut reduced by extra WHERE terms */ #define WHERE_OMIT_OFFSET 0x01000000 /* Set offset counter to zero */ | | > | 157684 157685 157686 157687 157688 157689 157690 157691 157692 157693 157694 157695 157696 157697 157698 157699 | #define WHERE_IN_EARLYOUT 0x00040000 /* Perhaps quit IN loops early */ #define WHERE_BIGNULL_SORT 0x00080000 /* Column nEq of index is BIGNULL */ #define WHERE_IN_SEEKSCAN 0x00100000 /* Seek-scan optimization for IN */ #define WHERE_TRANSCONS 0x00200000 /* Uses a transitive constraint */ #define WHERE_BLOOMFILTER 0x00400000 /* Consider using a Bloom-filter */ #define WHERE_SELFCULL 0x00800000 /* nOut reduced by extra WHERE terms */ #define WHERE_OMIT_OFFSET 0x01000000 /* Set offset counter to zero */ #define WHERE_COROUTINE 0x02000000 /* Implemented by co-routine. ** NB: False-negatives are possible */ #define WHERE_EXPRIDX 0x04000000 /* Uses an index-on-expressions */ #endif /* !defined(SQLITE_WHEREINT_H) */ /************** End of whereInt.h ********************************************/ /************** Continuing where we left off in wherecode.c ******************/ |
︙ | ︙ | |||
166456 166457 166458 166459 166460 166461 166462 166463 166464 166465 166466 166467 166468 166469 | #ifdef SQLITE_ENABLE_STAT4 pNew->rRun = rSize + 16 - 2*((pTab->tabFlags & TF_HasStat4)!=0); #else pNew->rRun = rSize + 16; #endif ApplyCostMultiplier(pNew->rRun, pTab->costMult); whereLoopOutputAdjust(pWC, pNew, rSize); rc = whereLoopInsert(pBuilder, pNew); pNew->nOut = rSize; if( rc ) break; }else{ Bitmask m; if( pProbe->isCovering ){ m = 0; | > > > > | 166462 166463 166464 166465 166466 166467 166468 166469 166470 166471 166472 166473 166474 166475 166476 166477 166478 166479 | #ifdef SQLITE_ENABLE_STAT4 pNew->rRun = rSize + 16 - 2*((pTab->tabFlags & TF_HasStat4)!=0); #else pNew->rRun = rSize + 16; #endif ApplyCostMultiplier(pNew->rRun, pTab->costMult); whereLoopOutputAdjust(pWC, pNew, rSize); if( pSrc->pSelect ){ if( pSrc->fg.viaCoroutine ) pNew->wsFlags |= WHERE_COROUTINE; pNew->u.btree.pOrderBy = pSrc->pSelect->pOrderBy; } rc = whereLoopInsert(pBuilder, pNew); pNew->nOut = rSize; if( rc ) break; }else{ Bitmask m; if( pProbe->isCovering ){ m = 0; |
︙ | ︙ | |||
167288 167289 167290 167291 167292 167293 167294 167295 167296 167297 167298 167299 167300 167301 | } } } whereLoopClear(db, pNew); return rc; } /* ** Examine a WherePath (with the addition of the extra WhereLoop of the 6th ** parameters) to see if it outputs rows in the requested ORDER BY ** (or GROUP BY) without requiring a separate sort operation. Return N: ** ** N>0: N terms of the ORDER BY clause are satisfied | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 167298 167299 167300 167301 167302 167303 167304 167305 167306 167307 167308 167309 167310 167311 167312 167313 167314 167315 167316 167317 167318 167319 167320 167321 167322 167323 167324 167325 167326 167327 167328 167329 167330 167331 167332 167333 167334 167335 167336 167337 167338 167339 167340 167341 167342 167343 167344 167345 167346 167347 167348 167349 167350 167351 167352 167353 167354 167355 167356 167357 167358 167359 167360 167361 167362 167363 167364 167365 167366 167367 167368 167369 167370 167371 167372 167373 167374 167375 167376 167377 167378 167379 167380 167381 167382 167383 167384 167385 167386 167387 167388 167389 167390 167391 167392 167393 167394 167395 167396 167397 167398 167399 167400 167401 167402 | } } } whereLoopClear(db, pNew); return rc; } /* Implementation of the order-by-subquery optimization: ** ** WhereLoop pLoop, which the iLoop-th term of the nested loop, is really ** a subquery or CTE that has an ORDER BY clause. See if any of the terms ** in the subquery ORDER BY clause will satisfy pOrderBy from the outer ** query. Mark off all satisfied terms (by setting bits in *pOBSat) and ** return TRUE if they do. If not, return false. ** ** Example: ** ** CREATE TABLE t1(a,b,c, PRIMARY KEY(a,b)); ** CREATE TABLE t2(x,y); ** WITH t3(p,q) AS MATERIALIZED (SELECT x+y, x-y FROM t2 ORDER BY x+y) ** SELECT * FROM t3 JOIN t1 ON a=q ORDER BY p, b; ** ** The CTE named "t3" comes out in the natural order of "p", so the first ** first them of "ORDER BY p,b" is satisfied by a sequential scan of "t3" ** and sorting only needs to occur on the second term "b". ** ** Limitations: ** ** (1) The optimization is not applied if the outer ORDER BY contains ** a COLLATE clause. The optimization might be applied if the ** outer ORDER BY uses NULLS FIRST, NULLS LAST, ASC, and/or DESC as ** long as the subquery ORDER BY does the same. But if the ** outer ORDER BY uses COLLATE, even a redundant COLLATE, the ** optimization is bypassed. ** ** (2) The subquery ORDER BY terms must exactly match subquery result ** columns, including any COLLATE annotations. This routine relies ** on iOrderByCol to do matching between order by terms and result ** columns, and iOrderByCol will not be set if the result column ** and ORDER BY collations differ. ** ** (3) The subquery and outer ORDER BY can be in opposite directions as ** long as the subquery is materialized. If the subquery is ** implemented as a co-routine, the sort orders must be in the same ** direction because there is no way to run a co-routine backwards. */ static SQLITE_NOINLINE int wherePathMatchSubqueryOB( WhereInfo *pWInfo, /* The WHERE clause */ WhereLoop *pLoop, /* The nested loop term that is a subquery */ int iLoop, /* Which level of the nested loop. 0==outermost */ int iCur, /* Cursor used by the this loop */ ExprList *pOrderBy, /* The ORDER BY clause on the whole query */ Bitmask *pRevMask, /* When loops need to go in reverse order */ Bitmask *pOBSat /* Which terms of pOrderBy are satisfied so far */ ){ int iOB; /* Index into pOrderBy->a[] */ int jSub; /* Index into pSubOB->a[] */ u8 rev = 0; /* True if iOB and jSub sort in opposite directions */ u8 revIdx = 0; /* Sort direction for jSub */ Expr *pOBExpr; /* Current term of outer ORDER BY */ ExprList *pSubOB; /* Complete ORDER BY on the subquery */ pSubOB = pLoop->u.btree.pOrderBy; assert( pSubOB!=0 ); for(iOB=0; (MASKBIT(iOB) & *pOBSat)!=0; iOB++){} for(jSub=0; jSub<pSubOB->nExpr && iOB<pOrderBy->nExpr; jSub++, iOB++){ if( pSubOB->a[jSub].u.x.iOrderByCol==0 ) break; pOBExpr = pOrderBy->a[iOB].pExpr; if( pOBExpr->op!=TK_COLUMN && pOBExpr->op!=TK_AGG_COLUMN ) break; if( pOBExpr->iTable!=iCur ) break; if( pOBExpr->iColumn!=pSubOB->a[jSub].u.x.iOrderByCol-1 ) break; if( (pWInfo->wctrlFlags & WHERE_GROUPBY)==0 ){ u8 sfOB = pOrderBy->a[iOB].fg.sortFlags; /* sortFlags for iOB */ u8 sfSub = pSubOB->a[jSub].fg.sortFlags; /* sortFlags for jSub */ if( (sfSub & KEYINFO_ORDER_BIGNULL) != (sfOB & KEYINFO_ORDER_BIGNULL) ){ break; } revIdx = sfSub & KEYINFO_ORDER_DESC; if( jSub>0 ){ if( (rev^revIdx)!=(sfOB & KEYINFO_ORDER_DESC) ){ break; } }else{ rev = revIdx ^ (sfOB & KEYINFO_ORDER_DESC); if( rev ){ if( (pLoop->wsFlags & WHERE_COROUTINE)!=0 ){ /* Cannot run a co-routine in reverse order */ break; } *pRevMask |= MASKBIT(iLoop); } } } *pOBSat |= MASKBIT(iOB); } return jSub>0; } /* ** Examine a WherePath (with the addition of the extra WhereLoop of the 6th ** parameters) to see if it outputs rows in the requested ORDER BY ** (or GROUP BY) without requiring a separate sort operation. Return N: ** ** N>0: N terms of the ORDER BY clause are satisfied |
︙ | ︙ | |||
167434 167435 167436 167437 167438 167439 167440 167441 167442 | testcase( pTerm->pExpr->op==TK_IS ); } obSat |= MASKBIT(i); } if( (pLoop->wsFlags & WHERE_ONEROW)==0 ){ if( pLoop->wsFlags & WHERE_IPK ){ pIndex = 0; nKeyCol = 0; | > > > > > > > > > > < | 167535 167536 167537 167538 167539 167540 167541 167542 167543 167544 167545 167546 167547 167548 167549 167550 167551 167552 167553 167554 167555 167556 167557 167558 167559 167560 | testcase( pTerm->pExpr->op==TK_IS ); } obSat |= MASKBIT(i); } if( (pLoop->wsFlags & WHERE_ONEROW)==0 ){ if( pLoop->wsFlags & WHERE_IPK ){ if( pLoop->u.btree.pOrderBy && OptimizationEnabled(db, SQLITE_OrderBySubq) && wherePathMatchSubqueryOB(pWInfo,pLoop,iLoop,iCur, pOrderBy,pRevMask, &obSat) ){ nColumn = 0; isOrderDistinct = 0; }else{ nColumn = 1; } pIndex = 0; nKeyCol = 0; }else if( (pIndex = pLoop->u.btree.pIndex)==0 || pIndex->bUnordered ){ return 0; }else{ nKeyCol = pIndex->nKeyCol; nColumn = pIndex->nColumn; assert( nColumn==nKeyCol+1 || !HasRowid(pIndex->pTable) ); assert( pIndex->aiColumn[nColumn-1]==XN_ROWID |
︙ | ︙ | |||
167531 167532 167533 167534 167535 167536 167537 | } if( iColumn==XN_EXPR ){ isOrderDistinct = 0; } } /* Find the ORDER BY term that corresponds to the j-th column | | | 167641 167642 167643 167644 167645 167646 167647 167648 167649 167650 167651 167652 167653 167654 167655 | } if( iColumn==XN_EXPR ){ isOrderDistinct = 0; } } /* Find the ORDER BY term that corresponds to the j-th column ** of the index and mark that ORDER BY term having been satisfied. */ isMatch = 0; for(i=0; bOnce && i<nOrderBy; i++){ if( MASKBIT(i) & obSat ) continue; pOBExpr = sqlite3ExprSkipCollateAndLikely(pOrderBy->a[i].pExpr); testcase( wctrlFlags & WHERE_GROUPBY ); testcase( wctrlFlags & WHERE_DISTINCTBY ); |
︙ | ︙ | |||
237270 237271 237272 237273 237274 237275 237276 | int bDesc; /* Iterate in descending rowid order */ int nPhrase; /* Number of phrases in expression */ Fts5ExprPhrase **apExprPhrase; /* Pointers to phrase objects */ }; /* ** eType: | | > > > > | 237380 237381 237382 237383 237384 237385 237386 237387 237388 237389 237390 237391 237392 237393 237394 237395 237396 237397 237398 237399 237400 237401 237402 237403 237404 | int bDesc; /* Iterate in descending rowid order */ int nPhrase; /* Number of phrases in expression */ Fts5ExprPhrase **apExprPhrase; /* Pointers to phrase objects */ }; /* ** eType: ** Expression node type. Usually one of: ** ** FTS5_AND (nChild, apChild valid) ** FTS5_OR (nChild, apChild valid) ** FTS5_NOT (nChild, apChild valid) ** FTS5_STRING (pNear valid) ** FTS5_TERM (pNear valid) ** ** An expression node with eType==0 may also exist. It always matches zero ** rows. This is created when a phrase containing no tokens is parsed. ** e.g. "". ** ** iHeight: ** Distance from this node to furthest leaf. This is always 0 for nodes ** of type FTS5_STRING and FTS5_TERM. For all other nodes it is one ** greater than the largest child value. */ struct Fts5ExprNode { |
︙ | ︙ | |||
240328 240329 240330 240331 240332 240333 240334 240335 240336 240337 240338 240339 240340 240341 | } } static int fts5ExprCheckPoslists(Fts5ExprNode *pNode, i64 iRowid){ pNode->iRowid = iRowid; pNode->bEof = 0; switch( pNode->eType ){ case FTS5_TERM: case FTS5_STRING: return (pNode->pNear->apPhrase[0]->poslist.n>0); case FTS5_AND: { int i; for(i=0; i<pNode->nChild; i++){ | > | 240442 240443 240444 240445 240446 240447 240448 240449 240450 240451 240452 240453 240454 240455 240456 | } } static int fts5ExprCheckPoslists(Fts5ExprNode *pNode, i64 iRowid){ pNode->iRowid = iRowid; pNode->bEof = 0; switch( pNode->eType ){ case 0: case FTS5_TERM: case FTS5_STRING: return (pNode->pNear->apPhrase[0]->poslist.n>0); case FTS5_AND: { int i; for(i=0; i<pNode->nChild; i++){ |
︙ | ︙ | |||
252387 252388 252389 252390 252391 252392 252393 | } } return pRet; } static void fts5ApiPhraseNext( | | < > > > > | | 252502 252503 252504 252505 252506 252507 252508 252509 252510 252511 252512 252513 252514 252515 252516 252517 252518 252519 252520 252521 252522 252523 252524 252525 252526 252527 252528 252529 252530 252531 252532 | } } return pRet; } static void fts5ApiPhraseNext( Fts5Context *pCtx, Fts5PhraseIter *pIter, int *piCol, int *piOff ){ if( pIter->a>=pIter->b ){ *piCol = -1; *piOff = -1; }else{ int iVal; pIter->a += fts5GetVarint32(pIter->a, iVal); if( iVal==1 ){ /* Avoid returning a (*piCol) value that is too large for the table, ** even if the position-list is corrupt. The caller might not be ** expecting it. */ int nCol = ((Fts5Table*)(((Fts5Cursor*)pCtx)->base.pVtab))->pConfig->nCol; pIter->a += fts5GetVarint32(pIter->a, iVal); *piCol = (iVal>=nCol ? nCol-1 : iVal); *piOff = 0; pIter->a += fts5GetVarint32(pIter->a, iVal); } *piOff += (iVal-2); } } |
︙ | ︙ | |||
253115 253116 253117 253118 253119 253120 253121 | static void fts5SourceIdFunc( sqlite3_context *pCtx, /* Function call context */ int nArg, /* Number of args */ sqlite3_value **apUnused /* Function arguments */ ){ assert( nArg==0 ); UNUSED_PARAM2(nArg, apUnused); | | | 253233 253234 253235 253236 253237 253238 253239 253240 253241 253242 253243 253244 253245 253246 253247 | static void fts5SourceIdFunc( sqlite3_context *pCtx, /* Function call context */ int nArg, /* Number of args */ sqlite3_value **apUnused /* Function arguments */ ){ assert( nArg==0 ); UNUSED_PARAM2(nArg, apUnused); sqlite3_result_text(pCtx, "fts5: 2024-08-16 18:51:46 7a0cdc7edb704a88a77b748cd28f6e00c49849cc2c1af838b95b34232ecc21f9", -1, SQLITE_TRANSIENT); } /* ** Return true if zName is the extension on one of the shadow tables used ** by this module. */ static int fts5ShadowName(const char *zName){ |
︙ | ︙ |
Changes to extsrc/sqlite3.h.
︙ | ︙ | |||
144 145 146 147 148 149 150 | ** ** See also: [sqlite3_libversion()], ** [sqlite3_libversion_number()], [sqlite3_sourceid()], ** [sqlite_version()] and [sqlite_source_id()]. */ #define SQLITE_VERSION "3.47.0" #define SQLITE_VERSION_NUMBER 3047000 | | | 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 | ** ** See also: [sqlite3_libversion()], ** [sqlite3_libversion_number()], [sqlite3_sourceid()], ** [sqlite_version()] and [sqlite_source_id()]. */ #define SQLITE_VERSION "3.47.0" #define SQLITE_VERSION_NUMBER 3047000 #define SQLITE_SOURCE_ID "2024-08-16 18:51:46 7a0cdc7edb704a88a77b748cd28f6e00c49849cc2c1af838b95b34232ecc21f9" /* ** CAPI3REF: Run-Time Library Version Numbers ** KEYWORDS: sqlite3_version sqlite3_sourceid ** ** These interfaces provide the same information as the [SQLITE_VERSION], ** [SQLITE_VERSION_NUMBER], and [SQLITE_SOURCE_ID] C preprocessor macros |
︙ | ︙ |