From 2f419ec1a6546d6f4b3492a6cdf0bf3796526c5c Mon Sep 17 00:00:00 2001 From: Fabrizio Ferri-Benedetti Date: Fri, 20 Oct 2023 08:57:04 +0200 Subject: [PATCH 1/6] Update framework-splunk_otel_java_agent.md --- docs/framework-splunk_otel_java_agent.md | 35 +++++++++++++----------- 1 file changed, 19 insertions(+), 16 deletions(-) diff --git a/docs/framework-splunk_otel_java_agent.md b/docs/framework-splunk_otel_java_agent.md index e8b438376b..ef5fb1a10d 100644 --- a/docs/framework-splunk_otel_java_agent.md +++ b/docs/framework-splunk_otel_java_agent.md @@ -1,9 +1,8 @@ # Splunk Distribution of OpenTelemetry Java Instrumentation -The Splunk OpenTelemetry Java Agent buildpack framework will cause an application to be automatically instrumented -with the [Splunk distribution of OpenTelemetry Java Instrumentation](https://github.com/signalfx/splunk-otel-java). - -Trace data will be sent directly to Splunk Observability Cloud. +This buildpack framework automatically instruments your Java application +with the [Splunk distribution of OpenTelemetry Java Instrumentation](https://github.com/signalfx/splunk-otel-java) +to send trace data to Splunk Observability Cloud. @@ -16,15 +15,17 @@ Trace data will be sent directly to Splunk Observability Cloud.
-Tags are printed to standard output by the buildpack detect script +The buildpack detect script prints tags to standard output. ## User-Provided Service -Users are currently expected to `create-user-provided-service` (cups) of the collector -and bind it to their application. The service MUST contain the string `splunk-o11y`. +Users are currently expected to provide their own `user-provided-service` (cups) of the collector +and bind it to their application. + +The service name MUST contain the string `splunk-o11y`. For example, to create a service named `splunk-o11y` that represents Observability Cloud -realm `us0` and represents a user environment named `cf-demo`, you could use the following +realm `us0` and represents a user environment named `cf-demo`, use the following commands: ``` @@ -34,27 +35,29 @@ $ cf bind-service myApp splunk-o11y $ cf restage myApp ``` -The `credential` field of the service should provide these entries: +Provide the following values using the `credential` field of the service: | Name | Required? | Description |------------------------|-----------| ----------- -| `splunk.access.token` | Yes | The Splunk [org access token](https://docs.splunk.com/observability/admin/authentication-tokens/org-tokens.html). -| `splunk.realm` | Yes | The Splunk realm where data will be sent. This is commonly `us0` or `eu0` etc. -| `otel.*` or `splunk.*` | Optional | All additional credentials starting with these prefixes will be appended to the application's JVM arguments as system properties. +| `splunk.access.token` | Yes | Splunk [org access token](https://docs.splunk.com/observability/admin/authentication-tokens/org-tokens.html). +| `splunk.realm` | Yes | Splunk realm where data will be sent. This is commonly `us0`, `eu0`, and so on. See [Available regions or realms](https://docs.splunk.com/observability/en/get-started/service-description.html#available-regions-or-realms) for more information. +| `otel.*` or `splunk.*` | Optional | All additional credentials starting with these prefixes are appended to the application's JVM arguments as system properties. ### Choosing a version -Most users should skip this and simply use the latest version of the agent available (the default). -To override the default and choose a specific version, you can use the `JBP_CONFIG_*` mechanism +To override the default and choose a specific version, use the `JBP_CONFIG_*` mechanism and set the `JBP_CONFIG_SPLUNK_OTEL_JAVA_AGENT` environment variable for your application. -For example, to use version 1.16.0 of the Splunk OpenTelemetry Java Instrumentation, you -could run: +For example, to use version 1.16.0 of the Splunk OpenTelemetry Java Instrumentation, run: + ``` $ cf set-env testapp JBP_CONFIG_SPLUNK_OTEL_JAVA_AGENT '{version: 1.16.0}' ``` + +In most cases you can use the latest or default version of the agent available. # Additional Resources * [Splunk Observability](https://www.splunk.com/en_us/products/observability.html) +* [Official documentation of the Splunk Java agent](https://docs.splunk.com/observability/en/gdi/get-data-in/application/java/get-started.html) * [Splunk Distribution of OpenTelemetry Java](https://github.com/signalfx/splunk-otel-java) on GitHub From b044b987cc0053cb29ce8981f630ff76302cdde0 Mon Sep 17 00:00:00 2001 From: Konstantin Semenov Date: Fri, 3 Feb 2023 18:28:11 +0000 Subject: [PATCH 2/6] experiment: GCP CloudSQL certificates integration --- .../framework/cloud_sql_security_provider.rb | 131 ++++++++++++++++++ 1 file changed, 131 insertions(+) create mode 100644 lib/java_buildpack/framework/cloud_sql_security_provider.rb diff --git a/lib/java_buildpack/framework/cloud_sql_security_provider.rb b/lib/java_buildpack/framework/cloud_sql_security_provider.rb new file mode 100644 index 0000000000..c2df0952f8 --- /dev/null +++ b/lib/java_buildpack/framework/cloud_sql_security_provider.rb @@ -0,0 +1,131 @@ +# frozen_string_literal: true + +# Cloud Foundry Java Buildpack +# Copyright 2013-2020 the original author or authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +require 'fileutils' +require 'shellwords' +require 'tempfile' +require 'java_buildpack/component/versioned_dependency_component' +require 'java_buildpack/framework' +require 'java_buildpack/util/qualify_path' + +module JavaBuildpack + module Framework + + # Encapsulates the functionality for enabling zero-touch Safenet ProtectApp Java Security Provider support. + class CloudSqlSecurityProvider < JavaBuildpack::Component::VersionedDependencyComponent + include JavaBuildpack::Util + + # (see JavaBuildpack::Component::BaseComponent#compile) + def compile + download_zip false + + @droplet.copy_resources + + credentials = @application.services.find_service(FILTER, 'sslrootcert', 'sslcert', 'sslkey')['credentials'] + + pkcs12 = merge_client_credentials credentials + add_client_credentials pkcs12 + + add_trusted_certificates credentials['sslrootcert'] + end + + # (see JavaBuildpack::Component::BaseComponent#release) + def release + java_opts = @droplet.java_opts + + add_additional_properties(java_opts) + end + + protected + + # (see JavaBuildpack::Component::VersionedDependencyComponent#supports?) + def supports? + @application.services.one_service? FILTER, 'sslrootcert', 'sslcert', 'sslkey' + end + + private + + FILTER = /csb-google-mysql/.freeze + + private_constant :FILTER + + def add_additional_properties(java_opts) + java_opts + .add_system_property('javax.net.ssl.keyStore', keystore) + .add_system_property('javax.net.ssl.keyStorePassword', password) + end + + def add_client_credentials(pkcs12) + shell "#{keytool} -importkeystore -noprompt -destkeystore #{keystore} -deststorepass #{password} " \ + "-srckeystore #{pkcs12.path} -srcstorepass #{password} -srcstoretype pkcs12" \ + " -alias #{File.basename(pkcs12)}" + end + + def add_trusted_certificates(trusted_certificate) + File.open("#{@droplet.root}/ssl/certs/ca-certificates.crt", 'a') do |f| + f.write("#{trusted_certificate}\n") + end + end + + def ext_dir + @droplet.sandbox + 'ext' + end + + def keystore + @droplet.sandbox + 'cloud-sql-keystore.jks' + end + + def keytool + @droplet.java_home.root + 'bin/keytool' + end + + def merge_client_credentials(credentials) + certificate = write_certificate credentials['sslcert'] + private_key = write_private_key credentials['sslkey'] + + pkcs12 = Tempfile.new('pkcs12-') + pkcs12.close + + shell "openssl pkcs12 -export -in #{certificate.path} -inkey #{private_key.path} " \ + "-name #{File.basename(pkcs12)} -out #{pkcs12.path} -passout pass:#{password}" + + pkcs12 + end + + def password + 'cloud-sql-keystore-password' + end + + def write_certificate(certificate) + Tempfile.open('certificate-') do |f| + f.write "#{certificate}\n" + f.sync + f + end + end + + def write_private_key(private_key) + Tempfile.open('private-key-') do |f| + f.write "#{private_key}\n" + f.sync + f + end + end + + end + end +end From 7e1faa950fd872d5385b4176041d344e91b6445b Mon Sep 17 00:00:00 2001 From: Konstantin Semenov Date: Fri, 3 Feb 2023 18:58:34 +0000 Subject: [PATCH 3/6] experiment: add logging --- .../framework/cloud_sql_security_provider.rb | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/lib/java_buildpack/framework/cloud_sql_security_provider.rb b/lib/java_buildpack/framework/cloud_sql_security_provider.rb index c2df0952f8..eabf688c6e 100644 --- a/lib/java_buildpack/framework/cloud_sql_security_provider.rb +++ b/lib/java_buildpack/framework/cloud_sql_security_provider.rb @@ -31,6 +31,7 @@ class CloudSqlSecurityProvider < JavaBuildpack::Component::VersionedDependencyCo # (see JavaBuildpack::Component::BaseComponent#compile) def compile + log '#release'.yellow download_zip false @droplet.copy_resources @@ -45,6 +46,7 @@ def compile # (see JavaBuildpack::Component::BaseComponent#release) def release + log '#release'.yellow java_opts = @droplet.java_opts add_additional_properties(java_opts) @@ -54,6 +56,7 @@ def release # (see JavaBuildpack::Component::VersionedDependencyComponent#supports?) def supports? + log '#supports?'.yellow @application.services.one_service? FILTER, 'sslrootcert', 'sslcert', 'sslkey' end @@ -63,6 +66,10 @@ def supports? private_constant :FILTER + + def log(message) + puts "#{'===========>'.blue} #{'CloudSqlSecurityProvider'.red.bold} #{message}" + end def add_additional_properties(java_opts) java_opts .add_system_property('javax.net.ssl.keyStore', keystore) From 090d4efbb276ea97ed991c6a3d989fe34926efc6 Mon Sep 17 00:00:00 2001 From: Konstantin Semenov Date: Fri, 3 Feb 2023 19:04:53 +0000 Subject: [PATCH 4/6] Fix CA certificate path --- lib/java_buildpack/framework/cloud_sql_security_provider.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/java_buildpack/framework/cloud_sql_security_provider.rb b/lib/java_buildpack/framework/cloud_sql_security_provider.rb index eabf688c6e..5ce8098618 100644 --- a/lib/java_buildpack/framework/cloud_sql_security_provider.rb +++ b/lib/java_buildpack/framework/cloud_sql_security_provider.rb @@ -83,7 +83,7 @@ def add_client_credentials(pkcs12) end def add_trusted_certificates(trusted_certificate) - File.open("#{@droplet.root}/ssl/certs/ca-certificates.crt", 'a') do |f| + File.open("#{@droplet.root}/etc/ssl/certs/ca-certificates.crt", 'a') do |f| f.write("#{trusted_certificate}\n") end end From 01d1c6205bc7324793f323cd44e3abb767850bb3 Mon Sep 17 00:00:00 2001 From: Konstantin Semenov Date: Fri, 3 Feb 2023 19:18:54 +0000 Subject: [PATCH 5/6] Added security provider to the list --- config/cloud_sql_security_provider.yml | 18 +++++++++++ config/components.yml | 1 + .../framework/cloud_sql_security_provider.rb | 32 ++++++++++--------- .../cloud_sql_security_provider/index.yml | 0 4 files changed, 36 insertions(+), 15 deletions(-) create mode 100644 config/cloud_sql_security_provider.yml create mode 100644 resources/cloud_sql_security_provider/index.yml diff --git a/config/cloud_sql_security_provider.yml b/config/cloud_sql_security_provider.yml new file mode 100644 index 0000000000..b5c97e95ef --- /dev/null +++ b/config/cloud_sql_security_provider.yml @@ -0,0 +1,18 @@ +# Cloud Foundry Java Buildpack +# Copyright 2013-2020 the original author or authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Configuration for the CloudSql Security Provider framework +--- +enabled: true \ No newline at end of file diff --git a/config/components.yml b/config/components.yml index 90a12ec28e..af10f5b4b1 100644 --- a/config/components.yml +++ b/config/components.yml @@ -49,6 +49,7 @@ frameworks: - "JavaBuildpack::Framework::ClientCertificateMapper" - "JavaBuildpack::Framework::ContainerCustomizer" - "JavaBuildpack::Framework::ContainerSecurityProvider" + - "JavaBuildpack::Framework::CloudSqlSecurityProvider" - "JavaBuildpack::Framework::ContrastSecurityAgent" - "JavaBuildpack::Framework::DatadogJavaagent" - "JavaBuildpack::Framework::Debug" diff --git a/lib/java_buildpack/framework/cloud_sql_security_provider.rb b/lib/java_buildpack/framework/cloud_sql_security_provider.rb index 5ce8098618..740d5de42b 100644 --- a/lib/java_buildpack/framework/cloud_sql_security_provider.rb +++ b/lib/java_buildpack/framework/cloud_sql_security_provider.rb @@ -18,21 +18,20 @@ require 'fileutils' require 'shellwords' require 'tempfile' -require 'java_buildpack/component/versioned_dependency_component' +require 'java_buildpack/component/base_component' require 'java_buildpack/framework' require 'java_buildpack/util/qualify_path' module JavaBuildpack module Framework - # Encapsulates the functionality for enabling zero-touch Safenet ProtectApp Java Security Provider support. - class CloudSqlSecurityProvider < JavaBuildpack::Component::VersionedDependencyComponent + # Encapsulates the functionality for enabling secure communication with GCP CloudSQL instances. + class CloudSqlSecurityProvider < JavaBuildpack::Component::BaseComponent include JavaBuildpack::Util # (see JavaBuildpack::Component::BaseComponent#compile) def compile - log '#release'.yellow - download_zip false + return unless supports? @droplet.copy_resources @@ -41,22 +40,26 @@ def compile pkcs12 = merge_client_credentials credentials add_client_credentials pkcs12 - add_trusted_certificates credentials['sslrootcert'] + add_trusted_certificate credentials['sslrootcert'] end # (see JavaBuildpack::Component::BaseComponent#release) def release - log '#release'.yellow + return unless supports? + java_opts = @droplet.java_opts add_additional_properties(java_opts) end + def detect + CloudSqlSecurityProvider.to_s.dash_case + end + protected # (see JavaBuildpack::Component::VersionedDependencyComponent#supports?) def supports? - log '#supports?'.yellow @application.services.one_service? FILTER, 'sslrootcert', 'sslcert', 'sslkey' end @@ -67,9 +70,6 @@ def supports? private_constant :FILTER - def log(message) - puts "#{'===========>'.blue} #{'CloudSqlSecurityProvider'.red.bold} #{message}" - end def add_additional_properties(java_opts) java_opts .add_system_property('javax.net.ssl.keyStore', keystore) @@ -82,10 +82,12 @@ def add_client_credentials(pkcs12) " -alias #{File.basename(pkcs12)}" end - def add_trusted_certificates(trusted_certificate) - File.open("#{@droplet.root}/etc/ssl/certs/ca-certificates.crt", 'a') do |f| - f.write("#{trusted_certificate}\n") - end + def add_trusted_certificate(trusted_certificate) + cert = Tempfile.new('ca-cert-') + cert.write(trusted_certificate) + cert.close + + shell "#{keytool} -import -trustcacerts -cacerts -storepass changeit -noprompt -alias CloudSQLCA -file #{cert.path}" end def ext_dir diff --git a/resources/cloud_sql_security_provider/index.yml b/resources/cloud_sql_security_provider/index.yml new file mode 100644 index 0000000000..e69de29bb2 From 9cac2c4be006312ef5c8a218a91160caa7486986 Mon Sep 17 00:00:00 2001 From: Konstantin Semenov Date: Fri, 3 Feb 2023 22:16:57 +0000 Subject: [PATCH 6/6] [#184278537](https://www.pivotaltracker.com/story/show/184278537) --- .../framework/cloud_sql_security_provider.rb | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/lib/java_buildpack/framework/cloud_sql_security_provider.rb b/lib/java_buildpack/framework/cloud_sql_security_provider.rb index 740d5de42b..192adb19c1 100644 --- a/lib/java_buildpack/framework/cloud_sql_security_provider.rb +++ b/lib/java_buildpack/framework/cloud_sql_security_provider.rb @@ -58,14 +58,13 @@ def detect protected - # (see JavaBuildpack::Component::VersionedDependencyComponent#supports?) def supports? @application.services.one_service? FILTER, 'sslrootcert', 'sslcert', 'sslkey' end private - FILTER = /csb-google-mysql/.freeze + FILTER = /csb-google-/.freeze private_constant :FILTER @@ -90,10 +89,6 @@ def add_trusted_certificate(trusted_certificate) shell "#{keytool} -import -trustcacerts -cacerts -storepass changeit -noprompt -alias CloudSQLCA -file #{cert.path}" end - def ext_dir - @droplet.sandbox + 'ext' - end - def keystore @droplet.sandbox + 'cloud-sql-keystore.jks' end