Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Linking AWS MQ infrastructure entities #1898

Closed
wants to merge 46 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
2f660ab
Adding cloud.resource_id attribute support to spans/transaction traces
meiao Apr 18, 2024
d2f8364
Adding SpanFactory unit test
meiao Apr 19, 2024
96f7f16
Reverting where DynamoDB instrumentation test is ignored
meiao Apr 19, 2024
557bdd3
Disabling DynamoDB 1 tests
meiao Apr 22, 2024
2103555
Ignoring test in DynamoDB v1
meiao Apr 23, 2024
46a9cab
Merge branch 'dynamodb-arn' of github.com:newrelic/newrelic-java-agen…
jasonjkeller Apr 29, 2024
99977de
Add cloud.resource_id attribute to message produce/consume spans. Add…
jasonjkeller May 2, 2024
592a01d
Merge branch 'main' of github.com:newrelic/newrelic-java-agent into d…
jasonjkeller May 2, 2024
e049b95
Add host/port to external message api for amqp
obenkenobi May 2, 2024
7684192
Report message broker host/port
obenkenobi May 3, 2024
4e39b63
Fix compilation issue with new external message params
obenkenobi May 3, 2024
00e47d2
Improvements with verifier testing
obenkenobi May 3, 2024
4c38cef
Refactor rabbitmq modules so verifier & tests pass
obenkenobi May 6, 2024
d362c7a
Refactor internal amqp-1.7.2 nr package
obenkenobi May 6, 2024
fa8d5c6
Add activemq client instrumentation to get endpoint
obenkenobi May 6, 2024
bc0eb51
Set host and port in jms instrumentation
obenkenobi May 6, 2024
fda38a9
Set host and port in jms-3 instrumentation
obenkenobi May 6, 2024
a0c6aeb
Fix verifier issues with activemq
obenkenobi May 7, 2024
1231875
Fix where consume params are set
obenkenobi May 7, 2024
f3fac5a
Add message broker endpoint attributes to span factory
obenkenobi May 14, 2024
275c6c4
Add operation + destination to message rollup metric for endpoint
obenkenobi May 15, 2024
ff9d035
Change message broker host attribute to be message.broker.endpoint
obenkenobi May 16, 2024
39812ae
Switch to otel message attribute names
obenkenobi May 17, 2024
3b2a439
Alter mq endpoint for temp queues/topics
obenkenobi May 17, 2024
52e7c25
Merge from main
obenkenobi May 17, 2024
e348b2a
Improve caching for parsing hostname in activemq
obenkenobi May 17, 2024
7afaa8e
Add copyright headers
obenkenobi May 17, 2024
8281ba3
Remove span attributes
obenkenobi May 20, 2024
2c1c794
Change unit to instrumentation test for activemq
obenkenobi May 20, 2024
8edd405
Mockito verify asserts
obenkenobi May 20, 2024
75b8ce9
Change external api to pass host & port in one method
obenkenobi May 20, 2024
ef102b5
Update newrelic.yml
obenkenobi May 20, 2024
3d59d61
Simplify instance metric
obenkenobi May 21, 2024
a4e8bb5
Add new metrics for rabbitmq
obenkenobi May 21, 2024
8e0cd15
Move reporting amqp instance to PrivateApi
obenkenobi May 23, 2024
c23e6df
Report msg broker inst in PrivateApi & move amqp code to instr modules
obenkenobi May 24, 2024
cad8280
replace message instance metrics with spans
obenkenobi Jun 3, 2024
8a0aabf
Remove unneeded configuration
obenkenobi Jun 3, 2024
afbef10
Add test for agent attribute names for spans empty in tracer
obenkenobi Jun 3, 2024
aa8f02a
Add missing overrides for the private API
obenkenobi Jun 4, 2024
7ca0d40
Merge branch 'main' of github.com:newrelic/newrelic-java-agent into a…
obenkenobi Jun 4, 2024
2aa42d0
Fix graphql 22 compilation for new private API changes
obenkenobi Jun 4, 2024
fe5cc8f
Add otel compatible routing key attribute for RabbitMQ
obenkenobi Jun 5, 2024
2eab2bf
Add otel attributes for rabbitMQ queues and exchanges
obenkenobi Jun 5, 2024
5215be1
Set empty exchanges in "messaging.destination.name" attr to be "Default"
obenkenobi Jun 6, 2024
cae16b6
Set span.kind to be producer or consumer
obenkenobi Jun 17, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
package com.newrelic.agent.bridge;

import java.util.Map;
import java.util.function.Function;

/**
* Allows instrumentation and bridge API implementations to use collections from third partly libraries without
Expand All @@ -28,6 +29,16 @@ public interface CollectionFactory {
*/
<K, V> Map<K, V> createConcurrentWeakKeyedMap();

/**
* Wraps the provided function into one that will cache the results for future calls.
* @param loader the function that calculates the value.
* @param maxSize the max number of items to be cached.
* @return the cached item, or the result of the loader call.
* @param <K> the type of key
* @param <V> the type of value stored/returned
*/
<K, V> Function<K, V> memoize(Function<K, V> loader, int maxSize);

/**
* Create a time based eviction cache in which an entry's age is determined on a last-write basis.
*
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,33 @@
import java.util.HashMap;
import java.util.Map;
import java.util.WeakHashMap;
import java.util.concurrent.ConcurrentHashMap;
import java.util.function.Function;

/**
* This implementation of {@link CollectionFactory} will only be used if the agent-bridge
* is being used by an application and the agent is NOT being loaded. Thus, it is unlikely
* that the objects created by this implementation are going to receive much use.
* So methods in this implementation do not need to implement all functional requirements
* of the methods in the interface, but they should not break under low use.
*/
public class DefaultCollectionFactory implements CollectionFactory {

@Override
public <K, V> Map<K, V> createConcurrentWeakKeyedMap() {
return Collections.synchronizedMap(new WeakHashMap<K, V>());
}

@Override
public <K, V> Function<K, V> memoize(Function<K, V> loader, int maxSize) {
Map<K, V> map = new ConcurrentHashMap<>();
return k -> map.computeIfAbsent(k, k1 -> {
meiao marked this conversation as resolved.
Show resolved Hide resolved
if (map.size() >= maxSize) {
map.remove(map.keySet().iterator().next());
}
return loader.apply(k1);
});
}
/**
* Note: In this implementation, this method will return a simple concurrent map since an eviction
* cache can't be easily created with just vanilla JDK Map SDKs.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@

package com.newrelic.agent.bridge;

import com.newrelic.api.agent.DestinationType;

import javax.management.MBeanServer;
import java.io.Closeable;
import java.util.Map;
Expand Down Expand Up @@ -71,6 +73,11 @@ public void setInstanceName(String instanceName) {
public void addTracerParameter(String key, String value) {
}

@Override
public void addTracerParameter(String key, String value, boolean addToSpan) {

}

@Override
public void addTracerParameter(String key, Map<String, String> values) {
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@

package com.newrelic.agent.bridge;

import com.newrelic.api.agent.DestinationType;

import javax.management.MBeanServer;
import java.io.Closeable;
import java.util.Map;
Expand Down Expand Up @@ -57,6 +59,8 @@ public interface PrivateApi {

void addTracerParameter(String key, String value);

void addTracerParameter(String key, String value, boolean addToSpan);

void addTracerParameter(String key, Map<String, String> values);

/**
Expand Down Expand Up @@ -122,4 +126,5 @@ public interface PrivateApi {
*/
@Deprecated
void setInstanceName(String instanceName);

}
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ public class MessageConsumeParameters extends com.newrelic.api.agent.MessageCons
@Deprecated
protected MessageConsumeParameters(String library, DestinationType destinationType, String destinationName,
InboundHeaders inboundHeaders) {
super(library, destinationType.toApiDestinationType(), destinationName, inboundHeaders);
super(library, destinationType.toApiDestinationType(), destinationName, inboundHeaders, null, null, null);
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
public class MessageProduceParameters extends com.newrelic.api.agent.MessageProduceParameters implements ExternalParameters {

/**
* @Deprecated Do not use. Use {@link com.newrelic.api.agent.MessageProduceParameters#MessageProduceParameters} instead.
* @Deprecated Do not use. Use the fluent builder {@link com.newrelic.api.agent.MessageProduceParameters#library(String)} instead.
*
* @param library
* @param destinationType
Expand All @@ -26,7 +26,7 @@ public class MessageProduceParameters extends com.newrelic.api.agent.MessageProd
@Deprecated
protected MessageProduceParameters(String library, DestinationType destinationType, String destinationName,
OutboundHeaders outboundHeaders) {
super(library, destinationType.toApiDestinationType(), destinationName, outboundHeaders);
super(library, destinationType.toApiDestinationType(), destinationName, outboundHeaders, null, null, null);
}

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
/*
*
* * Copyright 2024 New Relic Corporation. All rights reserved.
* * SPDX-License-Identifier: Apache-2.0
*
*/
package com.newrelic.agent.bridge.messaging;

public class BrokerInstance {
private final String hostName;
private final Integer port;

public static BrokerInstance empty() {
return new BrokerInstance(null, null);
}

public BrokerInstance(String host, Integer port) {
this.hostName = host;
this.port = port;
}

public String getHostName() {
return hostName;
}

public Integer getPort() {
return port;
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
/*
*
* * Copyright 2024 New Relic Corporation. All rights reserved.
* * SPDX-License-Identifier: Apache-2.0
*
*/
package com.newrelic.agent.bridge.messaging;

public class JmsProperties {
public static final String NR_JMS_BROKER_INSTANCE_PROPERTY = "NR_JMS_BROKER_INSTANCE_PROPERTY";
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
/*
*
* * Copyright 2024 New Relic Corporation. All rights reserved.
* * SPDX-License-Identifier: Apache-2.0
*
*/

package com.newrelic.agent.util;

import com.newrelic.agent.bridge.AgentBridge;

import java.util.function.Function;


public class AwsAccountUtil {
private static AwsAccountUtil INSTANCE = AwsAccountUtil.create();

private final Function<String, Long> CACHE = AgentBridge.collectionFactory.memoize(this::doDecodeAccount, 32);

public static AwsAccountUtil get() {
return INSTANCE;
}

public Long decodeAccount(String accessKey) {
return CACHE.apply(accessKey);
}

private Long doDecodeAccount(String awsAccessKeyId) {
String accessKeyWithoutPrefix = awsAccessKeyId.substring(4).toLowerCase();
long encodedAccount = base32Decode(accessKeyWithoutPrefix);
// magic number
long mask = 140737488355200L;
// magic incantation to find out the account
return (encodedAccount & mask) >> 7;
}

/**
* Character range is A-Z, 2-7. 'A' being 0 and '7', 31.
* Characters outside of this range will be considered 0.
* @param src the string to be decoded. Must be at least 10 characters.
* @return a long containing first 6 bytes of the base 32 decoded data.
* @throws ArrayIndexOutOfBoundsException if src has less than 10 characters
*/
private long base32Decode(String src) {
long base = 0;
char[] chars = src.toCharArray();
// each char is 5 bits, we need 48 bits
for (int i = 0; i < 10; i++) {
char c = chars[i];
base <<= 5;
if (c >= 'a' && c <= 'z') {
base += c - 'a';
} else if (c >= '2' && c <= '7') {
base += c - '2' + 26;
}
}
// 50 bits were read, dropping the lowest 2
return base >> 2;
}

private AwsAccountUtil() {
// prevent instantiation of utility class
}

private static AwsAccountUtil create() {
return new AwsAccountUtil();
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
/*
*
* * Copyright 2024 New Relic Corporation. All rights reserved.
* * SPDX-License-Identifier: Apache-2.0
*
*/

package com.newrelic.agent.util;

import org.junit.Test;

import static org.junit.Assert.*;

public class AwsAccountUtilTest {

@Test
public void decodeAccount() {
Long accountId = AwsAccountUtil.get().decodeAccount("FKKY6RVFFB77ZZZZZZZZ");
assertEquals(999999999999L, accountId.longValue());

accountId = AwsAccountUtil.get().decodeAccount("FKKYQAAAAAAAZZZZZZZZ");
assertEquals(1L, accountId.longValue());
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@
import com.newrelic.agent.TransactionListener;
import com.newrelic.agent.bridge.AgentBridge;
import com.newrelic.agent.bridge.TransactionNamePriority;
import com.newrelic.agent.browser.BrowserConfigTest;
import com.newrelic.agent.config.AgentConfigImpl;
import com.newrelic.agent.config.ConfigConstant;
import com.newrelic.agent.config.Hostname;
import com.newrelic.agent.dispatchers.WebRequestDispatcher;
import com.newrelic.agent.environment.AgentIdentity;
import com.newrelic.agent.errors.ErrorService;
Expand All @@ -38,7 +38,6 @@
import com.newrelic.agent.tracers.servlet.MockHttpResponse;
import com.newrelic.agent.transaction.PriorityTransactionName;
import com.newrelic.agent.transaction.TransactionThrowable;
import com.newrelic.agent.util.Obfuscator;
import com.newrelic.api.agent.DatastoreParameters;
import com.newrelic.api.agent.DestinationType;
import com.newrelic.api.agent.ExtendedRequest;
Expand Down Expand Up @@ -77,13 +76,11 @@
import java.net.URISyntaxException;
import java.net.URL;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

/* (non-javadoc)
Expand All @@ -95,6 +92,7 @@ public class ApiTest implements TransactionListener {
ApiTestHelper apiTestHelper = new ApiTestHelper();
private static final String CAT_CONFIG_FILE = "configs/cross_app_tracing_test.yml";
private static final String HIGH_SECURITY_CONFIG_FILE = "configs/high_security_config.yml";
public static String HOSTNAME = Hostname.getHostname(ServiceFactory.getConfigService().getDefaultAgentConfig());
private static final ClassLoader CLASS_LOADER = ApiTest.class.getClassLoader();

@Before
Expand Down Expand Up @@ -2007,6 +2005,27 @@ public void testMessagingAPI() throws Exception {
}
}

@Test
public void testMessagingAPIWithHostAndPort() throws Exception {
// override default agent config to disabled distributed tracing and use CAT instead
EnvironmentHolder holder = setupEnvironmentHolder(CAT_CONFIG_FILE, "cat_enabled_dt_disabled_test");
MessagingTestServer server = new MessagingTestServer(8088);

try {
server.start();
runTestMessagingAPIWithHostAndPort();
String messageBrokerMetric = "MessageBroker/JMS/Queue/Consume/Temp";
Assert.assertTrue("The following metric should exist: " + messageBrokerMetric, apiTestHelper.tranStats.getScopedStats().getStatsMap().containsKey(messageBrokerMetric));
} catch (IOException e) {
e.printStackTrace();
Assert.fail();
} finally {
Transaction.clearTransaction();
server.closeAllConnections();
holder.close();
}
}

@Trace(dispatcher = true)
private void runTestMessagingAPI() {
URL myURL = null;
Expand Down Expand Up @@ -2048,6 +2067,48 @@ private void runTestMessagingAPI() {
}
}

@Trace(dispatcher = true)
private void runTestMessagingAPIWithHostAndPort() {
URL myURL = null;
try {
Thread.sleep(600);
myURL = new URL("http://localhost:8088");
HttpUriRequest request = RequestBuilder.get().setUri(myURL.toURI()).build();

ApiTestHelper.OutboundWrapper outboundRequestWrapper = new ApiTestHelper.OutboundWrapper(request, HeaderType.MESSAGE);

// MessageProducer
ExternalParameters messageProduceParameters = MessageProduceParameters
.library("JMS")
.destinationType(DestinationType.NAMED_QUEUE)
.destinationName("MessageDestination")
.outboundHeaders(outboundRequestWrapper)
.instance(myURL.getHost(), myURL.getPort())
.build();
NewRelic.getAgent().getTracedMethod().reportAsExternal(messageProduceParameters);

Assert.assertTrue(request.getHeaders("NewRelicID").length != 0);
Assert.assertTrue(request.getHeaders("NewRelicTransaction").length != 0);

CloseableHttpClient connection = HttpClientBuilder.create().build();
CloseableHttpResponse response = connection.execute(request);

// MessageConsumer
ExternalParameters messageResponseParameters = MessageConsumeParameters
.library("JMS")
.destinationType(DestinationType.TEMP_QUEUE)
.destinationName("MessageDestination")
.inboundHeaders(new ApiTestHelper.InboundWrapper(response, HeaderType.MESSAGE))
.build();
NewRelic.getAgent().getTracedMethod().reportAsExternal(messageResponseParameters);

Assert.assertTrue(response.getHeaders("NewRelicAppData").length != 0);
} catch (Exception e) {
e.printStackTrace();
Assert.fail();
}
}

@Test
public void testNotNull() {
Assert.assertNotNull(NewRelic.getAgent().getTransaction().getTracedMethod());
Expand Down
Loading
Loading