Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Fix database sync #11879

Open
wants to merge 57 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 41 commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
443765f
Refactor method signature
koppor Oct 3, 2024
f3921c2
Rename maps
koppor Oct 3, 2024
92a2287
Cleanup EntriesAddedEvent
koppor Oct 3, 2024
f0465a6
Clean up EntriesRemovedEvent
koppor Oct 3, 2024
95fe7f6
Disable save actions for SQL
koppor Oct 3, 2024
279da5b
Fix logger string
koppor Oct 3, 2024
e19bc1a
Add TODO
koppor Oct 3, 2024
90e067d
Remove MySQL and Oracle dependencies
koppor Oct 3, 2024
35d6411
Remove PullChangesFromSharedAction
koppor Oct 3, 2024
f4f40c6
Improve doc and code comments
koppor Oct 3, 2024
7e459be
Mark variable to be removed
koppor Oct 4, 2024
f901151
Fix class name
koppor Oct 4, 2024
f6a2fe5
Introduce BibEntry.ENTRY_LINK_SEPARATOR
koppor Oct 4, 2024
8a11cb9
Cleanup code in CitationKeyListener
koppor Oct 4, 2024
fd958b6
More Java 8
koppor Oct 4, 2024
fd7697b
Minor code updates
koppor Oct 4, 2024
770cef0
Begin: separate metadata sync with all-entry-sync
koppor Oct 4, 2024
2c8faa7
Use "ON CONFLICT" of PostgreSQL
koppor Oct 4, 2024
dcf605f
Remove non-used get
koppor Oct 4, 2024
55bb387
Fix language
koppor Oct 4, 2024
fd3c184
Fix module-info.java
koppor Oct 4, 2024
5c863dc
Switch from "external" Postgres to embedded Postgres
koppor Oct 4, 2024
7e0ba3e
Remove escape and escape_Table (and add indexes)
koppor Oct 4, 2024
407b307
Fix method name "isFiltered" and also link reasoning
koppor Oct 4, 2024
f19101b
Merge branch 'main' into fix-sync
Siedlerchr Oct 4, 2024
5a36849
Integrate PostgreSQLProcessor in DBMSProcessor
koppor Oct 4, 2024
c1dda85
Let pgConnection do the waiting work (not the Java thread)
koppor Oct 4, 2024
323118f
Compilefix
koppor Oct 4, 2024
9f099ed
Change signature
koppor Oct 4, 2024
98242cf
Improve variable name
koppor Oct 4, 2024
dd6fb1b
Fix package name (and class name) for notifications
koppor Oct 4, 2024
451c971
Reorder methods
koppor Oct 4, 2024
2577ac6
Switch from UUID to CUID2
koppor Oct 4, 2024
b17aa35
Reorder fields
koppor Oct 4, 2024
76da442
Remove unused method
koppor Oct 4, 2024
72ed0bb
Write "Id" (instead of ID)
koppor Oct 4, 2024
c1fbcc4
Rename "Revision" to "Version"
koppor Oct 4, 2024
5b83b84
Streamline insertEntries
koppor Oct 4, 2024
6c1c2b0
Begin to revine remote-storage-sql.md
koppor Oct 5, 2024
2e66e10
Merge branch 'fix-sync' of github.com:JabRef/jabref into fix-sync
koppor Oct 7, 2024
383c665
Merge branch 'main' into fix-sync
koppor Oct 7, 2024
5bc7abd
Some minor code comments and SQL optimization
koppor Oct 7, 2024
f919bbd
Merge branch 'main' into fix-sync
calixtus Oct 7, 2024
c901b84
fix compile errors
Siedlerchr Oct 8, 2024
d2afcb4
Merge remote-tracking branch 'upstream/main' into fix-sync
Siedlerchr Oct 8, 2024
cedafb1
Merge branch 'fix-sync' of github.com:JabRef/jabref into fix-sync
koppor Oct 8, 2024
d1fd3f0
WIP: notificatoin
koppor Oct 9, 2024
e4c7130
Merge branch 'main' into fix-sync
koppor Oct 9, 2024
598d83e
Fix compile error (and typo)
koppor Oct 9, 2024
a78afc9
Add databaseName as parameter
koppor Oct 10, 2024
7eb4b07
Compilefix
koppor Oct 10, 2024
a9a2708
Merge branch 'fix-sync' of github.com:JabRef/jabref into fix-sync
koppor Oct 10, 2024
bc68c19
Remove unnecessary JavaDoc
koppor Oct 10, 2024
dd64830
Fix tests
koppor Oct 10, 2024
801acdf
Merge branch 'main' into fix-sync
koppor Nov 7, 2024
1b896d0
Compile fix
koppor Nov 7, 2024
2dd2beb
Merge branch 'main' into fix-sync
koppor Nov 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 2 additions & 29 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -198,18 +198,9 @@ jobs:
databasetests:
name: Database tests
runs-on: ubuntu-latest
services:
postgres:
image: postgres:13-alpine
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: postgres
ports:
- 5432:5432
# needed because the postgres container does not provide a healthcheck
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
steps:
- name: Shutdown Ubuntu MySQL
run: sudo service mysql stop # Shutdown the Default MySQL to save memory, "sudo" is necessary, please do not remove it
- name: Checkout source
uses: actions/checkout@v4
with:
Expand All @@ -228,24 +219,6 @@ jobs:
run: ./gradlew databaseTest --rerun-tasks
env:
CI: "true"
DBMS: "postgresql"
- name: Shutdown Ubuntu MySQL
run: sudo service mysql stop # Shutdown the Default MySQL, "sudo" is necessary, please not remove it
- name: Start custom MySQL
uses: mirromutth/[email protected]
with:
host port: 3800
container port: 3307
character set server: 'utf8'
collation server: 'utf8_general_ci'
mysql version: '8.0'
mysql database: 'jabref'
mysql root password: 'root'
- name: Run tests on MySQL
run: ./gradlew databaseTest --rerun-tasks
env:
CI: "true"
DBMS: "mysql"

guitests:
name: GUI tests
Expand Down
3 changes: 2 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@
- We improved the performance when pasting and importing entries in an existing library. [#11843](https://github.com/JabRef/jabref/pull/11843)
- When fulltext search is selected but indexing is deactivated, a dialog is now shown asking if the user wants to enable indexing now [#9491](https://github.com/JabRef/jabref/issues/9491)
- We changed instances of 'Search Selected' to 'Search Pre-configured' in Web Search Preferences UI. [#11871](https://github.com/JabRef/jabref/pull/11871)
- We rewrote the [remote SQL database](https://docs.jabref.org/collaborative-work/sqldatabase) support. ⚠️database tables will be migrated. [#11879](https://github.com/JabRef/jabref/pull/11879)

Check failure on line 59 in CHANGELOG.md

View workflow job for this annotation

GitHub Actions / Markdown

Trailing spaces

CHANGELOG.md:59:194 MD009/no-trailing-spaces Trailing spaces [Expected: 0 or 2; Actual: 1] https://github.com/DavidAnson/markdownlint/blob/v0.35.0/doc/md009.md
- We added a new CSS style class `main-table` for the main table. [#11881](https://github.com/JabRef/jabref/pull/11881)

### Fixed
Expand Down Expand Up @@ -93,7 +94,7 @@
- We removed support for case-sensitive and exact search. [#11542](https://github.com/JabRef/jabref/pull/11542)
- We removed the description of search strings. [#11542](https://github.com/JabRef/jabref/pull/11542)
- We removed support for importing using the SilverPlatterImporter (`Record INSPEC`). [#11576](https://github.com/JabRef/jabref/pull/11576)

- We removed support for MySQL/MariaDB and Oracle.



Expand Down
27 changes: 3 additions & 24 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -205,18 +205,10 @@ dependencies {

implementation 'com.fasterxml:aalto-xml:1.3.3'

implementation group: 'org.mariadb.jdbc', name: 'mariadb-java-client', version: '2.7.9'

implementation 'org.postgresql:postgresql:42.7.4'

// Support unix socket connection types
implementation 'com.kohlschutter.junixsocket:junixsocket-core:2.10.1'
implementation 'com.kohlschutter.junixsocket:junixsocket-mysql:2.10.0'

implementation ('com.oracle.ojdbc:ojdbc10:19.3.0.0') {
// causing module issues
exclude module: 'oraclepki'
}

implementation ('com.google.guava:guava:33.1.0-jre') {
// TODO: Remove this as soon as https://github.com/google/guava/issues/2960 is fixed
Expand Down Expand Up @@ -370,6 +362,9 @@ dependencies {
// Even if "compileOnly" is used, IntelliJ always adds to module-info.java. To avoid issues during committing, we use "implementation" instead of "compileOnly"
implementation 'io.github.adr:e-adr:2.0.0-SNAPSHOT'

implementation 'io.zonky.test:embedded-postgres:2.0.7'
implementation enforcedPlatform('io.zonky.test.postgres:embedded-postgres-binaries-bom:17.0.0')

testImplementation 'io.github.classgraph:classgraph:4.8.176'
testImplementation 'org.junit.jupiter:junit-jupiter:5.11.0'
testImplementation 'org.junit.platform:junit-platform-launcher:1.10.3'
Expand Down Expand Up @@ -803,24 +798,8 @@ jlink {
uses 'kong.unirest.core.json.JsonEngine'
uses 'org.eclipse.jgit.lib.Signer'
uses 'org.eclipse.jgit.transport.SshSessionFactory'
uses 'org.mariadb.jdbc.LocalInfileInterceptor'
uses 'org.mariadb.jdbc.authentication.AuthenticationPlugin'
uses 'org.mariadb.jdbc.credential.CredentialPlugin'
uses 'org.mariadb.jdbc.tls.TlsSocketPlugin'

provides 'org.mariadb.jdbc.tls.TlsSocketPlugin' with 'org.mariadb.jdbc.internal.protocol.tls.DefaultTlsSocketPlugin'
provides 'java.sql.Driver' with 'org.postgresql.Driver'
provides 'org.mariadb.jdbc.authentication.AuthenticationPlugin' with 'org.mariadb.jdbc.internal.com.send.authentication.CachingSha2PasswordPlugin',
'org.mariadb.jdbc.internal.com.send.authentication.ClearPasswordPlugin',
'org.mariadb.jdbc.internal.com.send.authentication.Ed25519PasswordPlugin',
'org.mariadb.jdbc.internal.com.send.authentication.NativePasswordPlugin',
'org.mariadb.jdbc.internal.com.send.authentication.OldPasswordPlugin',
'org.mariadb.jdbc.internal.com.send.authentication.SendGssApiAuthPacket',
'org.mariadb.jdbc.internal.com.send.authentication.SendPamAuthPacket',
'org.mariadb.jdbc.internal.com.send.authentication.Sha256PasswordPlugin'
provides 'org.mariadb.jdbc.credential.CredentialPlugin' with 'org.mariadb.jdbc.credential.aws.AwsIamCredentialPlugin',
'org.mariadb.jdbc.credential.env.EnvCredentialPlugin',
'org.mariadb.jdbc.credential.system.PropertiesCredentialPlugin'
provides 'java.security.Provider' with 'org.bouncycastle.jce.provider.BouncyCastleProvider',
'org.bouncycastle.pqc.jcajce.provider.BouncyCastlePQCProvider'
provides 'kong.unirest.core.json.JsonEngine' with 'kong.unirest.modules.gson.GsonEngine';
Expand Down
Loading
Loading