-
Notifications
You must be signed in to change notification settings - Fork 735
Description
Describe your environment
OS: (e.g, MacOS)
Python version: (e.g., Python 3.9.6)
SDK version: (e.g., 1.33.1)
API version: (e.g., 1.33.1)
What happened?
Unhandled exception occurs when LogExporter.export()
is called and LogRecordProcessor
did not guard against exceptions, which causes exception to bubble up and crash program. It should be LogExporter
job to guard against that not LogRecordProcessor
.
Steps to Reproduce
Custom processor:
from opentelemetry.sdk._logs import LogRecordProcessor
class RawLogProcessor(LogRecordProcessor):
def __init__(self, exporter):
self.exporter = exporter
def emit(self, log_data):
self.exporter.export([log_data])
def shutdown(self):
if hasattr(self.exporter, 'shutdown'):
self.exporter.shutdown()
def force_flush(self, timeout_millis=30000):
if hasattr(self.exporter, 'force_flush'):
return self.exporter.force_flush(timeout_millis)
return True
Reproduction program:
from opentelemetry._logs import set_logger_provider
from opentelemetry.exporter.otlp.proto.http._log_exporter import OTLPLogExporter
from opentelemetry.sdk._logs import LoggerProvider, LoggingHandler
from raw_log_processor import RawLogProcessor
import logging
otlp_exporter = OTLPLogExporter(endpoint="http://10.255.255.1:4318/v1/logs", timeout=1)
logger_provider = LoggerProvider()
logger_provider.add_log_record_processor(RawLogProcessor(otlp_exporter))
set_logger_provider(logger_provider)
logger = logging.getLogger(__name__)
logger.addHandler(LoggingHandler(logger_provider=logger_provider))
logger.setLevel(logging.INFO)
logger.propagate = False
logger.info("test")
print("If you see this, the logging worked!")
Expected Result
According to OTEL principles https://opentelemetry.io/docs/specs/otel/error-handling/#basic-error-handling-principles, SDK should not throw unhandled exceptions. For example, if ConnectTimeout
occurs during OTLPLogExporter._export()
method, LogExporter
should handle it and not throw.
Actual Result
SDK does not handle exception which bubble up and cause program to crash.
Exception: requests.exceptions.ConnectTimeout: HTTPConnectionPool(host='10.255.255.1', port=4318): Max retries exceeded with url: /v1/logs (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x104316a30>, 'Connection to 10.255.255.1 timed out. (connect timeout=1)'))
Additional context
The fix would be to handle exceptions in each LogExporter
export method implementation.
Would you like to implement a fix?
Yes
Tip
React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1
or me too
, to help us triage it. Learn more here.