Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Triples not existent in SPARQL endpoint after loading from repository #16

Open
KonradHoeffner opened this issue Oct 15, 2021 · 3 comments

Comments

@KonradHoeffner
Copy link

The importer doesn't return any errors but the data does not exist when importing from https://github.com/snikproject/ontology/.

virtuoso-import-docker$ docker-compose up   
WARN[0000] The "GIT_NAME" variable is not set. Defaulting to a blank string. 
WARN[0000] The "GIT_EMAIL" variable is not set. Defaulting to a blank string. 
[+] Running 2/0
 ⠿ Container virtuoso-import-docker-virtuoso-1  Created                                                                                                  0.0s
 ⠿ Container virtuoso-import-docker-importer-1  Recreated                                                                                                0.1s
Attaching to virtuoso-import-docker-importer-1, virtuoso-import-docker-virtuoso-1
virtuoso-import-docker-virtuoso-1  | chmod: cannot access '/clean-logs.sh': No such file or directory
virtuoso-import-docker-virtuoso-1  | 
virtuoso-import-docker-virtuoso-1  | 		Fri Oct 15 2021
virtuoso-import-docker-virtuoso-1  | 15:47:23 { Loading plugin 1: Type `plain', file `wikiv' in `/usr/local/virtuoso-opensource/lib/virtuoso/hosting'
virtuoso-import-docker-virtuoso-1  | 15:47:23   WikiV version 0.6 from OpenLink Software
virtuoso-import-docker-virtuoso-1  | 15:47:23   Support functions for WikiV collaboration tool
virtuoso-import-docker-virtuoso-1  | 15:47:23   SUCCESS plugin 1: loaded from /usr/local/virtuoso-opensource/lib/virtuoso/hosting/wikiv.so }
virtuoso-import-docker-virtuoso-1  | 15:47:23 { Loading plugin 2: Type `plain', file `mediawiki' in `/usr/local/virtuoso-opensource/lib/virtuoso/hosting'
virtuoso-import-docker-virtuoso-1  | 15:47:23   MediaWiki version 0.1 from OpenLink Software
virtuoso-import-docker-virtuoso-1  | 15:47:23   Support functions for MediaWiki collaboration tool
virtuoso-import-docker-virtuoso-1  | 15:47:23   SUCCESS plugin 2: loaded from /usr/local/virtuoso-opensource/lib/virtuoso/hosting/mediawiki.so }
virtuoso-import-docker-virtuoso-1  | 15:47:23 { Loading plugin 3: Type `plain', file `creolewiki' in `/usr/local/virtuoso-opensource/lib/virtuoso/hosting'
virtuoso-import-docker-virtuoso-1  | 15:47:23   CreoleWiki version 0.1 from OpenLink Software
virtuoso-import-docker-virtuoso-1  | 15:47:23   Support functions for CreoleWiki collaboration tool
virtuoso-import-docker-virtuoso-1  | 15:47:23   SUCCESS plugin 3: loaded from /usr/local/virtuoso-opensource/lib/virtuoso/hosting/creolewiki.so }
virtuoso-import-docker-virtuoso-1  | 15:47:23 { Loading plugin 10: Type `plain', file `shapefileio' in `/usr/local/virtuoso-opensource/lib/virtuoso/hosting'
virtuoso-import-docker-virtuoso-1  | 15:47:23   ShapefileIO version 0.1virt71 from OpenLink Software
virtuoso-import-docker-virtuoso-1  | 15:47:23   Shapefile support based on Frank Warmerdam's Shapelib
virtuoso-import-docker-virtuoso-1  | 15:47:23   SUCCESS plugin 10: loaded from /usr/local/virtuoso-opensource/lib/virtuoso/hosting/shapefileio.so }
virtuoso-import-docker-virtuoso-1  | 15:47:23 OpenLink Virtuoso Universal Server
virtuoso-import-docker-virtuoso-1  | 15:47:23 Version 07.20.3233-pthreads for Linux as of Jun 30 2021
virtuoso-import-docker-virtuoso-1  | 15:47:23 uses OpenSSL 1.0.2n  7 Dec 2017
virtuoso-import-docker-virtuoso-1  | 15:47:23 uses parts of PCRE, Html Tidy
virtuoso-import-docker-virtuoso-1  | 15:47:23 Database version 3126
virtuoso-import-docker-virtuoso-1  | 15:47:23 SQL Optimizer enabled (max 1000 layouts)
virtuoso-import-docker-virtuoso-1  | 15:47:24 Compiler unit is timed at 0.000073 msec
virtuoso-import-docker-virtuoso-1  | 15:47:24 Roll forward started
virtuoso-import-docker-virtuoso-1  | 15:47:24 Roll forward complete
virtuoso-import-docker-virtuoso-1  | 15:47:24 Checkpoint started
virtuoso-import-docker-virtuoso-1  | 15:47:24 Checkpoint finished, log reused
virtuoso-import-docker-virtuoso-1  | 15:47:26 HTTP/WebDAV server online at 8890
virtuoso-import-docker-virtuoso-1  | 15:47:26 Server online at 1111 (pid 1)
virtuoso-import-docker-importer-1  | CRON_JOB=import
virtuoso-import-docker-importer-1  | On branch master
virtuoso-import-docker-importer-1  | Your branch is up to date with 'origin/master'.
virtuoso-import-docker-importer-1  | 
virtuoso-import-docker-importer-1  | nothing to commit, working tree clean
virtuoso-import-docker-importer-1  | [INFO] Update repository (git pull) ...
virtuoso-import-docker-importer-1  | Already up to date.
virtuoso-import-docker-importer-1  | [INFO] git repo cloned/pulled. Continue with import.
virtuoso-import-docker-importer-1  | total 3.9M
virtuoso-import-docker-importer-1  | drwxr-xr-x 6 root root 4.0K Oct 15 14:51 .
virtuoso-import-docker-importer-1  | drwxr-xr-x 1 root root 4.0K Oct 15 15:47 ..
virtuoso-import-docker-importer-1  | drwxr-xr-x 8 root root 4.0K Oct 15 15:51 .git
virtuoso-import-docker-importer-1  | -rw-r--r-- 1 root root    6 Oct 15 14:51 .gitignore
virtuoso-import-docker-importer-1  | -rw-r--r-- 1 root root  21K Oct 15 14:51 LICENSE
virtuoso-import-docker-importer-1  | -rw-r--r-- 1 root root 1.2M Oct 15 14:51 bb.ttl
virtuoso-import-docker-importer-1  | -rw-r--r-- 1 root root  79K Oct 15 14:51 ciox.ttl
virtuoso-import-docker-importer-1  | -rwxr-xr-x 1 root root  150 Oct 15 14:51 combine
virtuoso-import-docker-importer-1  | -rw-r--r-- 1 root root 1.9K Oct 15 14:51 datapackage.json
virtuoso-import-docker-importer-1  | -rw-r--r-- 1 root root 895K Oct 15 14:51 he.ttl
virtuoso-import-docker-importer-1  | -rw-r--r-- 1 root root 115K Oct 15 14:51 it4it.ttl
virtuoso-import-docker-importer-1  | drwxr-xr-x 4 root root 4.0K Oct 15 14:51 limes
virtuoso-import-docker-importer-1  | -rw-r--r-- 1 root root 114K Oct 15 14:51 limes.nt
virtuoso-import-docker-importer-1  | -rw-r--r-- 1 root root  48K Oct 15 14:51 match.nt
virtuoso-import-docker-importer-1  | -rw-r--r-- 1 root root  13K Oct 15 14:51 meta.ttl
virtuoso-import-docker-importer-1  | -rw-r--r-- 1 root root 1.3M Oct 15 14:51 ob.ttl
virtuoso-import-docker-importer-1  | -rw-r--r-- 1 root root  81K Oct 15 14:51 persian.ttl
virtuoso-import-docker-importer-1  | -rw-r--r-- 1 root root  853 Oct 15 14:51 prefix.ttl
virtuoso-import-docker-importer-1  | -rw-r--r-- 1 root root 5.0K Oct 15 14:51 readme.md
virtuoso-import-docker-importer-1  | drwxr-xr-x 2 root root 4.0K Oct 15 14:51 script
virtuoso-import-docker-importer-1  | drwxr-xr-x 2 root root 4.0K Oct 15 14:51 sparql
virtuoso-import-docker-importer-1  | [INFO] waiting for store to come online
virtuoso-import-docker-importer-1  | REPORT
virtuoso-import-docker-importer-1  | VARCHAR
virtuoso-import-docker-importer-1  | _______________________________________________________________________________
virtuoso-import-docker-importer-1  | 
virtuoso-import-docker-importer-1  | OpenLink Virtuoso  Server
virtuoso-import-docker-importer-1  | Version 07.20.3233-pthreads for Linux as of Jun 30 2021 
virtuoso-import-docker-importer-1  | Started on: 2021-10-15 15:47 GMT+0
virtuoso-import-docker-importer-1  |  
virtuoso-import-docker-importer-1  | Database Status:
virtuoso-import-docker-importer-1  |   File size 69206016, 8448 pages, 6201 free.
virtuoso-import-docker-importer-1  |   20000 buffers, 705 used, 0 dirty 0 wired down, repl age 0 0 w. io 0 w/crsr.
virtuoso-import-docker-importer-1  |   Disk Usage: 735 reads avg 0 msec, 0% r 0% w last  0 s, 107 writes flush          0 MB/s,
virtuoso-import-docker-importer-1  |     23 read ahead, batch = 14.  Autocompact 0 in 0 out, 0% saved.
virtuoso-import-docker-importer-1  | Gate:  152 2nd in reads, 0 gate write waits, 0 in while read 0 busy scrap. 
virtuoso-import-docker-importer-1  | Log = /usr/local/virtuoso-opensource/var/lib/virtuoso/db/virtuoso.trx, 185 bytes
virtuoso-import-docker-importer-1  | 2162 pages have been changed since last backup (in checkpoint state)
virtuoso-import-docker-importer-1  | Current backup timestamp: 0x0000-0x00-0x00
virtuoso-import-docker-importer-1  | Last backup date: unknown
virtuoso-import-docker-importer-1  | Clients: 1 connects, max 1 concurrent
virtuoso-import-docker-importer-1  | RPC: 4 calls, 1 pending, 1 max until now, 0 queued, 0 burst reads (0%), 0 second 0M large, 8M max
virtuoso-import-docker-importer-1  | Checkpoint Remap 57 pages, 0 mapped back. 0 s atomic time.
virtuoso-import-docker-importer-1  |     DB master 8448 total 6201 free 57 remap 0 mapped back
virtuoso-import-docker-importer-1  |    temp  256 total 251 free
virtuoso-import-docker-importer-1  |  
virtuoso-import-docker-importer-1  | Lock Status: 0 deadlocks of which 0 2r1w, 0 waits,
virtuoso-import-docker-importer-1  |    Currently 1 threads running 0 threads waiting 0 threads in vdb.
virtuoso-import-docker-importer-1  | Pending:
virtuoso-import-docker-importer-1  |  
virtuoso-import-docker-importer-1  | Client 1111:1:  Account: dba, 200 bytes in, 286 bytes out, 1 stmts.
virtuoso-import-docker-importer-1  | PID: 26, OS: unix, Application: unknown, IP#: 172.18.0.3
virtuoso-import-docker-importer-1  | Transaction status: PENDING, 1 threads.
virtuoso-import-docker-importer-1  | Locks: 
virtuoso-import-docker-importer-1  |  
virtuoso-import-docker-importer-1  |  
virtuoso-import-docker-importer-1  | Running Statements:
virtuoso-import-docker-importer-1  |  Time (msec) Text
virtuoso-import-docker-importer-1  |            5 status()
virtuoso-import-docker-importer-1  |  
virtuoso-import-docker-importer-1  |  
virtuoso-import-docker-importer-1  | Hash indexes
virtuoso-import-docker-importer-1  |  
virtuoso-import-docker-importer-1  | 
virtuoso-import-docker-importer-1  | 37 Rows. -- 6 msec.
virtuoso-import-docker-importer-1  | 
virtuoso-import-docker-importer-1  | [INFO] registring RDF documents for import
virtuoso-import-docker-importer-1  | [INFO] now loading the following files: -rw-r--r-- 1 root root 114K Oct 15 14:51 limes.nt
virtuoso-import-docker-importer-1  | -rw-r--r-- 1 root root  48K Oct 15 14:51 match.nt
virtuoso-import-docker-importer-1  | 
virtuoso-import-docker-importer-1  | Done. -- 0 msec.
virtuoso-import-docker-importer-1  | 
virtuoso-import-docker-importer-1  | ls: cannot access '*.nq': No such file or directory
virtuoso-import-docker-importer-1  | 
virtuoso-import-docker-importer-1  | Done. -- 0 msec.
virtuoso-import-docker-importer-1  | 
virtuoso-import-docker-importer-1  | ls: cannot access '*.owl': No such file or directory
virtuoso-import-docker-importer-1  | 
virtuoso-import-docker-importer-1  | Done. -- 0 msec.
virtuoso-import-docker-importer-1  | 
virtuoso-import-docker-importer-1  | ls: cannot access '*.rdf': No such file or directory
virtuoso-import-docker-importer-1  | 
virtuoso-import-docker-importer-1  | Done. -- 0 msec.
virtuoso-import-docker-importer-1  | 
virtuoso-import-docker-importer-1  | ls: cannot access '*.trig': No such file or directory
virtuoso-import-docker-importer-1  | 
virtuoso-import-docker-importer-1  | Done. -- 0 msec.
virtuoso-import-docker-importer-1  | 
virtuoso-import-docker-importer-1  | [INFO] now loading the following files: -rw-r--r-- 1 root root 1.2M Oct 15 14:51 bb.ttl
virtuoso-import-docker-importer-1  | -rw-r--r-- 1 root root  79K Oct 15 14:51 ciox.ttl
virtuoso-import-docker-importer-1  | -rw-r--r-- 1 root root 895K Oct 15 14:51 he.ttl
virtuoso-import-docker-importer-1  | -rw-r--r-- 1 root root 115K Oct 15 14:51 it4it.ttl
virtuoso-import-docker-importer-1  | -rw-r--r-- 1 root root  13K Oct 15 14:51 meta.ttl
virtuoso-import-docker-importer-1  | -rw-r--r-- 1 root root 1.3M Oct 15 14:51 ob.ttl
virtuoso-import-docker-importer-1  | -rw-r--r-- 1 root root  81K Oct 15 14:51 persian.ttl
virtuoso-import-docker-importer-1  | -rw-r--r-- 1 root root  853 Oct 15 14:51 prefix.ttl
virtuoso-import-docker-importer-1  | 
virtuoso-import-docker-importer-1  | Done. -- 0 msec.
virtuoso-import-docker-importer-1  | 
virtuoso-import-docker-importer-1  | ls: cannot access '*.xml': No such file or directory
virtuoso-import-docker-importer-1  | 
virtuoso-import-docker-importer-1  | Done. -- 0 msec.
virtuoso-import-docker-importer-1  | 
virtuoso-import-docker-importer-1  | ls: cannot access '*.gz': No such file or directory
virtuoso-import-docker-importer-1  | 
virtuoso-import-docker-importer-1  | Done. -- 0 msec.
virtuoso-import-docker-importer-1  | 
virtuoso-import-docker-importer-1  | [INFO] deactivating auto-indexing
virtuoso-import-docker-importer-1  | 
virtuoso-import-docker-importer-1  | Done. -- 0 msec.
virtuoso-import-docker-importer-1  | 
virtuoso-import-docker-importer-1  | [INFO] Starting load process...
virtuoso-import-docker-virtuoso-1  | 15:51:04 PL LOG: Loader started
virtuoso-import-docker-virtuoso-1  | 15:51:04 PL LOG: No more files to load. Loader has finished,
@KonradHoeffner KonradHoeffner changed the title Triples not existant in SPARQL endpoint after loading from repository Triples not existent in SPARQL endpoint after loading from repository Oct 15, 2021
@KonradHoeffner
Copy link
Author

When I got to http://localhost:8890/sparql/ after the above, select count(*) where {?s ?p ?o.} only results in 5641 default triples but not the more than 80000 in the repository.

@KonradHoeffner
Copy link
Author

I think the problem is this:

Note: if the third parameter (graph_iri) of ld_dir() or ld_dir_all() is NULL, any data files that do not have a corresponding .graph file will not be loaded..

Source: http://vos.openlinksw.com/owiki/wiki/VOS/VirtBulkRDFLoader

Because I don't have any graph files, the data is not loaded but there is no warning or error message.

@KonradHoeffner
Copy link
Author

I suggest adding to the readme file that the import only works when there is a x.graph file for each file x or alternatively change the code so that it there is no graph file all triples get imported to the default graph.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant