Commit Graph

276 Commits

Author SHA1 Message Date
Evan Miller
fb960553e6 New mdb_set_bind_size function overrides MDB_BIND_SIZE
This should fix long-standing complaints about the default bind size
without causing undue memory inflation in existing applications.

Could make this adjustable on the command line later.

Supersedes:

https://github.com/mdbtools/mdbtools/pull/137
2020-09-02 14:30:07 -04:00
Evan Miller
b7dd44d0d4 Merge pull request #42 from evanmiller/mdb-exportjson
mdb-json tool
2020-09-02 12:05:02 -04:00
Evan Miller
0f5bbdf96c Rename tool to mdb-json and use modern option parsing 2020-09-02 11:47:50 -04:00
Evan Miller
01711b49f4 Merge branch 'hexbinaryexportmode' of github.com:davidhicks/mdbtools into export-hex 2020-09-02 09:53:50 -04:00
Evan Miller
a598148afc Merge branch 'export-mongodb-json' of github.com:rutsky/mdbtools into mdb-exportjson 2020-09-02 09:18:52 -04:00
Evan Miller
9a0abb43ae Fix GCC warnings 2020-09-01 22:43:10 -04:00
Evan Miller
c76fad6356 Add ancient mdb-queries tool by Leonard Leblanc
Modified to run with modern libmdb.

See: https://github.com/mdbtools/mdbtools/issues/122
2020-09-01 22:34:55 -04:00
Evan Miller
cea7553e00 Export fakeglib symbols from main library 2020-08-31 13:27:08 -04:00
Evan Miller
be888e0dd7 Support fuzz testing (#4)
Quickstart (requires Clang 6 or later):

$ export LIB_FUZZING_ENGINE=/path/to/fuzzing/library.a
$ ./configure --enable-fuzz-testing
$ make
$ cd src/fuzz
$ make fuzz_mdb
$ ./fuzz_mdb

Also add a new `mdb_open_buffer function` to facilitate in-memory
fuzz-testing. This requires fmemopen, which may not be present on all
systems. The internal API has been reworked to use file streams instead
of file descriptors. This allows reading from memory and reading from
files using a consistent API.
2020-08-31 13:03:58 -04:00
Evan Miller
90ee7bfcb5 Remove references to dmalloc
There are more modern tools for memory debugging, get rid of DMALLOC
crap in the source code.

I've left one reference in backend.c to prevent a merge conflict but
this can be removed later.
2020-08-20 08:59:08 -04:00
Evan Miller
5d078838ed [API break] Make date/boolean formats thread safe
Store the preferred date and boolean formats in the MdbHandle rather
than in global memory.
2020-08-19 21:26:06 -04:00
Evan Miller
3c6c8e07db Make mdb-count case-insensitive to table names
This matches the behavior of mdb-export. Thanks to Nicholas Crookston.
2020-08-13 06:32:41 -04:00
Nyall Dawson
1639283779 Fix int/guint comparison warnings 2020-08-12 13:04:20 +10:00
Evan Miller
d4f7bd6ec5 Remove mdb-check
This program is not built by default, and its functionality is covered
by the other utilties.
2020-08-10 06:58:05 -04:00
Evan Miller
7690ef376f Turn off -Wsign-compare for the lexer
Flex doesn't like it.
2020-08-09 16:23:06 -04:00
Evan Miller
c93156626c Fix all sign-compare warnings
Also flip on -Wsign-compare in the default configuration
2020-08-09 16:15:20 -04:00
Evan Miller
18de0070af New --disable-glib configure option
This option uses the fake GLib shim even when GLib is present on the
system.
2020-08-05 20:36:20 -04:00
Evan Miller
68d3a22e77 Merge branch 'master' into no-glib 2020-08-05 18:49:24 -04:00
Evan Miller
cac3f7aca5 Default to single quotes when exporting to Postgres
See:

brianb/mdbtools#119
brianb/mdbtools#128
2020-08-05 18:27:48 -04:00
Evan Miller
ce9ad2920d Merge branch 'master' into no-glib 2020-08-05 17:10:10 -04:00
Evan Miller
cde53c4332 Respect octal flag in mdb-export brianb/mdbtools#153
The *col_val <= 0 check appears to have been intended to print ASCII
characters as-is. But this is not a good check, and goes against the
documented behavior.
2020-08-05 13:38:50 -04:00
Evan Miller
7436fa0e0e Yet another attempted cygwin fix 2020-08-05 09:53:41 -04:00
Evan Miller
bcc6defe94 Merge branch 'master' into no-glib 2020-08-04 23:53:50 -04:00
Evan Miller
679c90141f Improve error message 2020-08-04 15:10:23 -04:00
Evan Miller
65b3701d67 Fix building utilities with GLib present 2020-08-04 08:02:10 -04:00
Evan Miller
e9f4c6c786 Remove GLib dependency WIP
Add a shim implementing half-assed versions of most of the GLib
functions used by MDB Tools. If GLib is detected at compile-time,
use it, otherwise use the shim.

This work is not complete, as the option-parsing code is not yet
implemented - so most of the command-line tools crash.
2020-08-03 17:19:40 -04:00
Evan Miller
a2fa34c0cb Merge https://github.com/Kaukov/mdbtools 2020-08-03 16:55:40 -04:00
Evan Miller
61d928ea68 Fix warning when compiling without Readline 2020-08-03 15:13:11 -04:00
Peter Kaukov
55858b6fe1 Fix flex library linking 2019-07-13 23:56:32 +03:00
VJ
c8fd67bf84 Fix more warnings as a result of #14 2019-05-31 21:59:37 -04:00
Evan Miller
0e9c1810f1 Improved support for Access 2010, 2013, and 2016
Some Access 2010 files use 0x03 as the version number rather than
0x0103. For this reason I have changed the call to mdb_get_int32 to
mdb_get_byte.

In addition, according to the Library of Congress page:

https://www.loc.gov/preservation/digital/formats/fdd/fdd000463.shtml

Access 2016 uses 0x05 as the version number. I have inferred from the
Wikipedia page that Access 2013 likely uses 0x04.
2018-03-05 06:39:57 -05:00
Cyber Emissary
3a245f13d5 Merge branch 'mdb-count' of https://github.com/kodonnell/mdbtools 2018-02-12 16:22:52 -05:00
Cyber Emissary
278f9ba441 Implemented bulk insert 2018-02-12 15:21:15 -05:00
Bruce Johnson
5bfd423e19 Update mdb_index_hash_text() call to match latest signature 2018-01-19 05:51:00 -05:00
kodonnell
b92f4cc882 added (trivial) mdb-count utility 2017-02-05 08:56:24 +13:00
Richard Mansfield
62f79ec3f3 Obey -e flag when quoting is disabled 2017-01-31 10:50:56 +13:00
Edward Abraham
4f10f09df8 Escape troublesome characters using a -e flag, following comment on issue #40 2017-01-26 17:26:01 +13:00
Brian Bruns
23bab7c2d6 add support for LIMIT clause in SQL engine 2016-08-30 08:43:34 -04:00
Brian Bruns
15230b5b73 Merge pull request #94 from ewen-naos-nz/boolean_data_as_words
mdb-export: Add boolean words option (TRUE/FALSE)
2016-08-29 09:40:51 -04:00
Joshua Short
e85c3438ec Adding option to for user-defined NULL token in CSV output (not sure if it should be escaped..) 2016-08-01 09:11:21 -07:00
Ewen McNeill
8b1db6c08e mdb-export: Add boolean words option (TRUE/FALSE)
Adds "-B" (--boolean-words) option to mdb-export, which will reconfigure
mdb/data.c to export TRUE/FALSE for boolean values instead of 1/0.  The
option is needed to support BOOLEAN fields on PostgreSQL, which will not
implicitly cast bare 1/0 into a BOOLEAN value.  Value literals are the
SQL TRUE/FALSE, and _quoted_ words meaning true/false and _quoted_
'1'/'0'.  With this flag the SQL TRUE/FALSE values are output, which should
work with several SQL databases.

PostgreSQL Reference:

http://www.postgresql.org/docs/current/static/datatype-boolean.html
2016-04-07 08:45:06 +12:00
Nirgal Vourgère
584119e415 Split bash-completion snippet in small ones
/usr/share/bash-completion/completions/ now uses dynamic loading based on file name.
2015-05-30 15:40:58 +02:00
William Rogers
82e3436494 Corrected memory being improperly freed
The memory for sql->bound_values is now freed by mdb_sql_reset, so these values no longer should be freed in dump_results_pp.  Also, this fixes a memory leak that was occurring when results were not being pretty printed.
2015-05-24 17:15:52 -05:00
William Rogers
9276aca347 Consolidated calls to mdb_sql_reset
SQL engine was being reset in both dump_results functions.  Rather than having several different functions responsible for resetting the engine depending on the circumstances, the reset now only occurs in run_query, the same function where the query is executed.
2015-05-24 17:05:41 -05:00
Vladimir Rutsky
3216c8de03 output all non-binary fields as unicode strings and binary fields as MongoDB extended JSON binary field
Now mdb-exportjson output can be successfully imported into MongoDB with
mongoimport even if:

 * columns have unicode names
 * there are fields of binary types (e.g. OLE)
2015-02-19 21:05:49 +03:00
Vladimir Rutsky
f894cc708f add base64 encoding implementation from Wikibooks
Source <https://en.wikibooks.org/wiki/Algorithm_Implementation/Miscellaneous/Base64>.

Licensed probably under CC-BY-SA. Compatibility with mdb-tools GPL
license should be checked.
2015-02-19 21:05:49 +03:00
Vladimir Rutsky
bc1483429d use JSON unicode escape \uUUUU instead \xXX for [0, 0x20) characters
JSON specification doesn't allow `\xXX` escape sequences
2015-02-19 21:05:49 +03:00
Vladimir Rutsky
47c0698609 treat only characters with codes in [0, 0x20) range as non-ascii 2015-02-19 21:05:49 +03:00
Vladimir Rutsky
d8cba08493 fix delimiter outputting when some fields values are missing 2015-02-19 20:51:57 +03:00
Vladimir Rutsky
b842480cd5 don't drop non-ascii characters by default
This corresponds to `-A` option description.
2015-02-19 20:48:44 +03:00