dcsimg
 
  en  
names in breadcrumbs

Import Log for GBIF data coverage

  • Started: May 28, 2024 11:12
  • Completed: 16:28:43
  • Failed: No.
  • Status: completed

Events (most recent first):

show warnings (21)
  • 16:28:43 (ends) resource.rb:225>fast.rb:177>pub_log.rb:73>import_log.rb:89= Complete
  • 16:28:43 (ends) delayed_job.rb:17>resource.rb:225>fast.rb:176>pub_log.rb:48= TOTAL TIME: 5.3h
  • 16:28:43 (starts) delayed_job.rb:17>resource.rb:225>fast.rb:163>pub_log.rb:48= TraitBank::Denormalizer.update_resource_vernaculars
  • 16:28:43 (starts) resource.rb:225>fast.rb:163>pub_log.rb:20>import_log.rb:82= Running
  • 16:28:29 (starts) delayed_job.rb:17>resource.rb:225>fast.rb:161>pub_log.rb:48= Resource#fix_native_nodes
  • 16:28:29 (starts) resource.rb:225>fast.rb:161>pub_log.rb:20>import_log.rb:82= Running
  • 16:28:29 (infos) delayed_job.rb:17>resource.rb:225>fast.rb:235>pub_log.rb:48= Removing /app/tmp/gbif_data_covera_node_ancestors.tsv
  • 16:28:29 (infos) delayed_job.rb:17>resource.rb:225>fast.rb:235>pub_log.rb:48= Removing /app/tmp/gbif_data_covera_scientific_names.tsv
  • 16:28:29 (infos) delayed_job.rb:17>resource.rb:225>fast.rb:235>pub_log.rb:48= Removing /app/tmp/gbif_data_covera_nodes.tsv
  • 16:28:28 (infos) resource.rb:225>fast.rb:334>slurp.rb:36>pub_log.rb:48= Removing trait and metadata files
  • 16:28:28 (infos) resource.rb:225>fast.rb:334>slurp.rb:351>pub_log.rb:48= Nodes: 1475149; Traits: 1402986; MetaData: 0
  • 16:28:20 (infos) resource.rb:225>fast.rb:334>slurp.rb:275>pub_log.rb:48= Importing 128000 rows from publish_metadata.tsv
  • 16:28:20 (infos) resource.rb:225>fast.rb:334>slurp.rb:592>pub_log.rb:48= adding new metadata
  • 16:28:20 (infos) resource.rb:225>fast.rb:334>slurp.rb:351>pub_log.rb:48= Nodes: 1475149; Traits: 1402986; MetaData: 0
  • 16:27:23 (infos) resource.rb:225>fast.rb:334>slurp.rb:275>pub_log.rb:48= Importing 128000 rows from publish_traits_chunk_15.csv
  • 16:26:55 (warns) fast.rb:334>each.rb:9>slurp.rb:289>pub_log.rb:48= ...re-trying.
  • 16:21:55 (warns) fast.rb:334>each.rb:9>slurp.rb:285>pub_log.rb:48= FAILED on build_nodes query (Page), will re-try 3 times after 5 minute pause (the site may be too busy to serve the CSV to Neo4j)...
  • 16:21:55 (warns) fast.rb:334>each.rb:9>slurp.rb:284>pub_log.rb:48= Cannot merge the following node because of null property value for 'page_id': (page:Page {page_id: null}) (Failure when processing file '/data/gbif_data_covera/publish_traits_chunk_14.csv' on line 59611 (which is the last row in the file).)
  • 16:21:55 (warns) fast.rb:334>each.rb:9>slurp.rb:571>pub_log.rb:48= Exception (Neo4j::Driver::Exceptions::ClientException) QUERY: {USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM 'https://eol.org/data/gbif_data_covera/publish_traits_chunk_14.csv' AS row WITH row WHERE 1=1 MERGE (page:Page { page_id: toInteger(row.page_id) })} MESSAGE: Cannot merge the following node because of null property value for 'page_id': (page:Page {page_id: null}) (Failure when processing file '/data/gbif_data_covera/publish_traits_chunk_14.csv' on line 59611 (which is the last row in the file).)
  • 16:21:50 (infos) fast.rb:334>each.rb:9>slurp.rb:275>pub_log.rb:48= Importing 128000 rows from publish_traits_chunk_14.csv
  • 15:53:49 (infos) fast.rb:334>each.rb:9>slurp.rb:329>pub_log.rb:48= Waiting 28 minutes for the part 14 of 15 to be added to neo4j.
  • 15:53:49 (infos) fast.rb:334>each.rb:9>slurp.rb:351>pub_log.rb:48= Nodes: 1413041; Traits: 1340878; MetaData: 0
  • 15:52:30 (infos) fast.rb:334>each.rb:9>slurp.rb:275>pub_log.rb:48= Importing 128000 rows from publish_traits_chunk_13.csv
  • 15:26:29 (infos) fast.rb:334>each.rb:9>slurp.rb:329>pub_log.rb:48= Waiting 26 minutes for the part 13 of 15 to be added to neo4j.
  • 15:26:29 (infos) fast.rb:334>each.rb:9>slurp.rb:351>pub_log.rb:48= Nodes: 1292492; Traits: 1225712; MetaData: 0
  • 15:25:06 (infos) fast.rb:334>each.rb:9>slurp.rb:275>pub_log.rb:48= Importing 128000 rows from publish_traits_chunk_12.csv
  • 15:01:05 (infos) fast.rb:334>each.rb:9>slurp.rb:329>pub_log.rb:48= Waiting 24 minutes for the part 12 of 15 to be added to neo4j.
  • 15:01:05 (infos) fast.rb:334>each.rb:9>slurp.rb:351>pub_log.rb:48= Nodes: 1183938; Traits: 1121702; MetaData: 0
  • 14:59:37 (infos) fast.rb:334>each.rb:9>slurp.rb:275>pub_log.rb:48= Importing 128000 rows from publish_traits_chunk_11.csv
  • 14:37:37 (infos) fast.rb:334>each.rb:9>slurp.rb:329>pub_log.rb:48= Waiting 22 minutes for the part 11 of 15 to be added to neo4j.
  • 14:37:37 (infos) fast.rb:334>each.rb:9>slurp.rb:351>pub_log.rb:48= Nodes: 1055938; Traits: 994847; MetaData: 0
  • 14:36:33 (infos) fast.rb:334>each.rb:9>slurp.rb:275>pub_log.rb:48= Importing 128000 rows from publish_traits_chunk_10.csv
  • 14:16:32 (infos) fast.rb:334>each.rb:9>slurp.rb:329>pub_log.rb:48= Waiting 20 minutes for the part 10 of 15 to be added to neo4j.
  • 14:16:32 (infos) fast.rb:334>each.rb:9>slurp.rb:351>pub_log.rb:48= Nodes: 1010240; Traits: 949149; MetaData: 0
  • 14:15:15 (infos) fast.rb:334>each.rb:9>slurp.rb:275>pub_log.rb:48= Importing 128000 rows from publish_traits_chunk_9.csv
  • 13:57:14 (infos) fast.rb:334>each.rb:9>slurp.rb:329>pub_log.rb:48= Waiting 18 minutes for the part 9 of 15 to be added to neo4j.
  • 13:57:14 (infos) fast.rb:334>each.rb:9>slurp.rb:351>pub_log.rb:48= Nodes: 882240; Traits: 821149; MetaData: 0
  • 13:55:53 (infos) fast.rb:334>each.rb:9>slurp.rb:275>pub_log.rb:48= Importing 128000 rows from publish_traits_chunk_8.csv
  • 13:39:52 (infos) fast.rb:334>each.rb:9>slurp.rb:329>pub_log.rb:48= Waiting 16 minutes for the part 8 of 15 to be added to neo4j.
  • 13:39:52 (infos) fast.rb:334>each.rb:9>slurp.rb:351>pub_log.rb:48= Nodes: 754240; Traits: 693149; MetaData: 0
  • 13:38:23 (infos) fast.rb:334>each.rb:9>slurp.rb:275>pub_log.rb:48= Importing 128000 rows from publish_traits_chunk_7.csv
  • 13:24:23 (infos) fast.rb:334>each.rb:9>slurp.rb:329>pub_log.rb:48= Waiting 14 minutes for the part 7 of 15 to be added to neo4j.
  • 13:24:23 (infos) fast.rb:334>each.rb:9>slurp.rb:351>pub_log.rb:48= Nodes: 626240; Traits: 565149; MetaData: 0
  • 13:23:22 (infos) fast.rb:334>each.rb:9>slurp.rb:275>pub_log.rb:48= Importing 128000 rows from publish_traits_chunk_6.csv
  • 13:11:21 (infos) fast.rb:334>each.rb:9>slurp.rb:329>pub_log.rb:48= Waiting 12 minutes for the part 6 of 15 to be added to neo4j.
  • 13:11:21 (infos) fast.rb:334>each.rb:9>slurp.rb:351>pub_log.rb:48= Nodes: 538444; Traits: 523756; MetaData: 0
  • 13:10:16 (warns) fast.rb:334>each.rb:9>slurp.rb:289>pub_log.rb:48= ...re-trying.
  • 13:05:16 (warns) fast.rb:334>each.rb:9>slurp.rb:285>pub_log.rb:48= FAILED on build_nodes query (Page), will re-try 3 times after 5 minute pause (the site may be too busy to serve the CSV to Neo4j)...
  • 13:05:16 (warns) fast.rb:334>each.rb:9>slurp.rb:284>pub_log.rb:48= Cannot merge the following node because of null property value for 'page_id': (page:Page {page_id: null}) (Failure when processing file '/data/gbif_data_covera/publish_traits_chunk_5.csv' on line 49632 (which is the last row in the file).)
  • 13:05:16 (warns) fast.rb:334>each.rb:9>slurp.rb:571>pub_log.rb:48= Exception (Neo4j::Driver::Exceptions::ClientException) QUERY: {USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM 'https://eol.org/data/gbif_data_covera/publish_traits_chunk_5.csv' AS row WITH row WHERE 1=1 MERGE (page:Page { page_id: toInteger(row.page_id) })} MESSAGE: Cannot merge the following node because of null property value for 'page_id': (page:Page {page_id: null}) (Failure when processing file '/data/gbif_data_covera/publish_traits_chunk_5.csv' on line 49632 (which is the last row in the file).)
EOL is hosted by: