Acoustic tag tracking data flow diagram #77
Comments
|
like, passive acoustic, or acoustic tags? if the latter, can I help with this? |
|
see https://ocean-tracking-network.github.io/node-manager-training/02_OTN_System_and%20_structure_and_Outputs%20/index.html for a discussion of the way OTN flows acoustic data from raw and unverified to verified and shareable. |
|
Hey @jdpye! Thanks for hopping in here. For this issue, I'm looking at acoustic tags (passive acoustic monitoring (PAM) data flow are starting to be drafted at ioos/mbon-docs#41). I'm trying to wrap my head around ATN's connection with OTN. @mmckinzie probably has thoughts to build out here. In looking at the first diagram on that page you referenced, what does the OBIS box mean? Is that an OBIS node that everything gets bundled into? Does OTN have something to pull from previous to DwC alignment? edit: clarifying the language for passive acoustics in this context |
|
The centralizing schema in the OTN database setup is named |
MathewBiddle
changed the title
|
Okay, so here is where I'm at. I'm having trouble figuring out how US base researchers would be able to archive at NOAA/NCEI as well as getting information into our federal discovery systems, but I think that's a conversation for another day. %%{
init: {
'theme': 'base',
'themeVariables': {
'primaryColor': '#007396',
'primaryTextColor': '#fff',
'primaryBorderColor': '#003087',
'lineColor': '#003087',
'secondaryColor': '#007396',
'tertiaryColor': '#CCD1D1'
},
'flowchart': { 'curve': 'basis' }
}
}%%
flowchart TD
A["Acoustic Tag"]
B[("OTN
Regional Acoustic
Node
or
OTN proper")]
C{{"Darwin Core
Alignment"}}
D[(NCEI)]
E[("OTN IPT
OBIS-oceantrack")]
F[/"MBON
Data Portal"\]
G([OBIS])
H([GBIF])
I[("IOOS Data Catalog
(data.ioos.us)")]
J[(NOAA OneStop)]
K[(data.gov)]
L[("Commerce
Data Hub")]
M[/"IOC-UNESCO Harmful Algae Information System"\]
N[/"Infographics"\]
O[/"ATN Data
Portal"\]
A --> B
B --> C
C --> E
E --> EP
B --OTN ERDDAP\nhttps://members.oceantrack.org/erddap/index.html\nor\nOTN node\ndatabase\nconnection\nor\nOTN GeoServer -->EP
G --> EP
H --> EP
E --> G
E --> H
D --> FC
I --> FC
subgraph EP [Example Products]
O
N
F
M
end
subgraph FC [Federal Catalogs]
J
K
L
end
click D "https://www.ncei.noaa.gov" "NCEI" _blank
click E "https://members.oceantrack.org/ipt/" "OTN IPT" _blank
click F "https://mbon.ioos.us" "MBON" _blank
click G "https://obis.org" "OBIS" _blank
click H "https://gbif.org" "GBIF" _blank
click I "https://data.ioos.us" "IOOS Catalog" _blank
click J "https://data.noaa.gov/onestop/" "NOAA OneStop" _blank
click K "https://data.gov" "data.gov" _blank
click O "https://portal.atn.ioos.us/" "ATN" _blank
|
|
After a chat with @jdpye some modifications to the flow in how it could look in some future... %%{
init: {
'theme': 'base',
'themeVariables': {
'primaryColor': '#007396',
'primaryTextColor': '#fff',
'primaryBorderColor': '#003087',
'lineColor': '#003087',
'secondaryColor': '#007396',
'tertiaryColor': '#CCD1D1'
},
'flowchart': { 'curve': 'basis' }
}
}%%
flowchart TD
A["Acoustic Tag"]
B[("OTN
Regional Acoustic
Node
or
OTN proper")]
C{{"Darwin Core
Alignment"}}
D[(NCEI)]
E[("OTN IPT
OBIS-oceantrack")]
P[("OTN ERDDAP\nor\nOTN node\ndatabase\nconnection\nor\nOTN GeoServer")]
F[/"MBON
Data Portal"\]
G([OBIS])
H([GBIF])
I[("IOOS Data Catalog
(data.ioos.us)")]
J[(NOAA OneStop)]
K[(data.gov)]
L[("Commerce
Data Hub")]
M[/"IOC-UNESCO Harmful Algae Information System"\]
N[/"Infographics"\]
O[/"ATN Data
Portal"\]
A --> B
B --> C
B .-> D
C --> E
E --> G
E --> H
D --> FC
I --> FC
FC .-> EP
E .-> EP
B -->P
P .-> EP
G .-> EP
H .-> EP
subgraph EP [Example Products]
O
N
F
M
end
subgraph FC [U.S. Federal Catalogs]
J
K
L
end
click D "https://www.ncei.noaa.gov" "NCEI" _blank
click E "https://members.oceantrack.org/ipt/" "OTN IPT" _blank
click P "https://members.oceantrack.org/erddap/index.html" _blank
click F "https://mbon.ioos.us" "MBON" _blank
click G "https://obis.org" "OBIS" _blank
click H "https://gbif.org" "GBIF" _blank
click I "https://data.ioos.us" "IOOS Catalog" _blank
click J "https://data.noaa.gov/onestop/" "NOAA OneStop" _blank
click K "https://data.gov" "data.gov" _blank
click O "https://portal.atn.ioos.us/" "ATN" _blank
|
|
@jdpye @mmckinzie could you please review the above diagram? Are there specific example products/applications we could highlight here for acoustic tracking? eg. remora? A note about linestyle, the dotted lines are hypothetical pathways. Solid lines are pathways that are currently functioning. |
|
I like having this diagram a lot! I would say there's been a lot of discussion and clamor for standing up an ATN Acoustic Data Mirror / FedNode between the regional OTN nodes and NCEI. And if we do that, it could in turn be the thing that feeds the ATN Data Portal, instead of the current state of affairs for ATN acoustics, which is acoustic project metadata coming from the OTN GeoServer and the OTN members portal site (via a scraper) |
|
I've got to revamp the ERDDAP situation a bit on the OTN side, but the GeoServer and the node-to-node connectivity are where things are happening in production right now. |
Like this? %%{
init: {
'theme': 'base',
'themeVariables': {
'primaryColor': '#007396',
'primaryTextColor': '#fff',
'primaryBorderColor': '#003087',
'lineColor': '#003087',
'secondaryColor': '#007396',
'tertiaryColor': '#CCD1D1'
},
'flowchart': { 'curve': 'basis' }
}
}%%
flowchart TD
A["Acoustic Tag"]
B[("OTN
Regional Acoustic
Node
or
OTN proper")]
Q[(ATN Acoustic Data Mirror\n/\nFedNode)]
C{{"Darwin Core
Alignment"}}
D[(NCEI)]
E[("OTN IPT
OBIS-oceantrack")]
P[("OTN ERDDAP\nor\nOTN node\ndatabase\nconnection\nor\nOTN GeoServer")]
F[/"MBON
Data Portal"\]
G([OBIS])
H([GBIF])
I[("IOOS Data Catalog
(data.ioos.us)")]
J[(NOAA OneStop)]
K[(data.gov)]
L[("Commerce
Data Hub")]
M[/"IOC-UNESCO Harmful Algae Information System"\]
N[/"Infographics"\]
O[/"ATN Data
Portal"\]
A --> B
B --> C
B .-> Q
Q .-> D
B .-> D
C --> E
E --> G
E --> H
D --> FC
I --> FC
FC .-> EP
E .-> EP
B -->P
P .-> EP
G .-> EP
H .-> EP
subgraph EP [Example Products]
O
N
F
M
end
subgraph FC [U.S. Federal Catalogs]
J
K
L
end
%% click D "https://www.ncei.noaa.gov" "NCEI" _blank
%% click E "https://members.oceantrack.org/ipt/" "OTN IPT" _blank
%% click P "https://members.oceantrack.org/erddap/index.html" _blank
%% click F "https://mbon.ioos.us" "MBON" _blank
%% click G "https://obis.org" "OBIS" _blank
%% click H "https://gbif.org" "GBIF" _blank
%% click I "https://data.ioos.us" "IOOS Catalog" _blank
%% click J "https://data.noaa.gov/onestop/" "NOAA OneStop" _blank
%% click K "https://data.gov" "data.gov" _blank
%% click O "https://portal.atn.ioos.us/" "ATN" _blank
I'm wondering if we need to split out OTN proper vs OTN regional node at this point? edit: removed hyperlinks in diagram edit2: reverted back to the original |
|
@MathewBiddle You could separate OTN proper from OTN regional node, but data will likely flow into ATN node from both sources depending on where researchers originally submitted data/metadata. So, for simplicity also fine to leave grouped together, in my opinion. Only other thoughts are that when we stand up a federated node, ATN DAC will likely preform the DwC conversion and serve data to OBIS and then serve decimated data from OBIS to NCEI?? for U.S. based researchers. I'd also add a dashed line connecting ATN node to the products box as we'll likely harvest data/metadata from ATN node to create project pages and/or other data product that can be served out through the Marine Life data network portals. |
|
How might we get acoustic tracking into the IOOS Data Catalog (https://data.ioos.us/)? I think we're sorting this out with satellite telemetry via ERDDAP which can be indexed by the IOOS catalog. One requirement for IOOS is data are represented in the IOOS Data Catalog... |
|
OTN has an ERDDAP server as well, and the old standard we used to publish data to it with was co-developed by US-IOOS. what sorts of metadata/data do you need to see in the catalog? |
|
@jdpye checkout @mwengren's comment at ioos/marine_life_data_network#52 (comment) ISO XML. I think an ERDDAP WAF of metadata records will suffice for those purposes. |
|
Oh we can get there no problem. ERDDAP or even GeoServer's catalog services for the web extension. |
|
I totally messed up the diagram above. Here is a re-rendering of my current understanding of what data flows might look like after tagging up with @jdpye: %%{
init: {
'theme': 'base',
'themeVariables': {
'primaryColor': '#007396',
'primaryTextColor': '#fff',
'primaryBorderColor': '#003087',
'lineColor': '#003087',
'secondaryColor': '#007396',
'tertiaryColor': '#CCD1D1'
},
'flowchart': { 'curve': 'basis' }
}
}%%
flowchart TD
A["Acoustic Tag"]
B[("OTN
Regional Node(s)
eg. ACT
FACT
GLATOS
etc")]
%% B[("OTN")]
%% Q[(ATN Acoustic Data Mirror/
%% FedNode??)]
C{{"Darwin Core
Alignment"}}
D[(NCEI)]
E[("OTN IPT
OBIS-oceantrack")]
P[("OTN ERDDAP
or
OTN node
database
connection
or
OTN GeoServer")]
G([OBIS])
H([GBIF])
I[("IOOS Data Catalog
(data.ioos.us)")]
J[(NOAA OneStop)]
K[(data.gov)]
L[("Commerce
Data Hub")]
A --> B
B -- Data Push --> C
%% B .-> Q
%% Q .-> D
B -. Data Push .-> D
C --> E
E --> G
E .-> H
D --> FC
I --> FC
B -->P
subgraph FC [U.S. Federal Catalogs]
J
K
L
end
|
a place to jot down what a data flow might look like for Acoustic tracking data.
The text was updated successfully, but these errors were encountered: