Need help for creating a custom log function to write to a glue job

Hi, I need help regarding this thing...how to set up a custom cloudwatch log to write to a specific glue job. I am facing some issue regarding this. Any input will be highly appreciated

14mo ago

ZestyPenguin

TCS

14mo

Try chatGpt

import boto3 import logging from datetime import datetime

Create a CloudWatch client

cloudwatch_logs = boto3.client('logs')

log_group = '/aws/glue/custom_log_group' # Custom log group name log_stream = 'glue_job_log_stream' # Define a stream name

Create log stream (if not exists)

try: cloudwatch_logs.create_log_stream(logGroupName=log_group, logStreamName=log_stream) except cloudwatch_logs.exceptions.ResourceAlreadyExistsException: pass

Create a logger instance

logger = logging.getLogger() logger.setLevel(logging.INFO)

Function to send logs to CloudWatch

def send_log_to_cloudwatch(message): timestamp = int(datetime.now().timestamp() * 1000) log_event = { 'logGroupName': log_group, 'logStreamName': log_stream, 'logEvents': [{ 'timestamp': timestamp, 'message': message }] }

# Send the log event response = cloudwatch_logs.put_log_events(**log_event) return response