For Benthos yaml / pipeline what we developed, we will call http API, to handle the http exception, we can catch the error thru error(), then send to error topic, but it seems that the detailed http error msg can not be saved as meta variable or directly save to root. It always roll back to null at output code.
Here it is the sample code
http: address: 0.0.0.0:4190 root_path: /benthosinput: kafka_franz: seed_broker: - seed-broker-server:15555 consumer_group: "consumer_group1" topic: in_topicpipeline: processors: - branch: processors: - mapping: | root = { "input_param1": "value1" }.key_values() - http: url: https://api.sampleserver.com/firstapi verb: POST - catch: - mapping: # meta errorMsg = error() root = { "error": error() } root = deleted() ...... # other work flow codeoutput: switch: cases: - check: errored() output: kafka_franz: seed_brokers: out_broker_server:16666 topic: "error_topic" key: $(! meta("kafka_key") processors: - log: level: INFO message: "Catch and send to error topic: $(! json() }" - output: ..... # other code to out topic
Since the above catch error() did not work, anyone has experience or suggested solution to keep the error() code, and send it to error topic.