์ƒ์„ธ ์ปจํ…์ธ 

๋ณธ๋ฌธ ์ œ๋ชฉ

[TIL] AWS Athena์—์„œ ์ฟผ๋ฆฌ ์ง„ํ–‰ ์ƒํƒœ ํ™•์ธํ•˜๊ธฐ

CLOUD/AWS

by ์—์Šคํ”„๋ฆฌํ„ฐ 2022. 10. 5. 21:09

๋ณธ๋ฌธ

728x90
๐Ÿ’ก Today I Learned ์š”์•ฝ (24ํšŒ์ฐจ)
- boto3์—์„  Athena ์ฟผ๋ฆฌ ์ƒํƒœ๋ฅผ ํ™•์ธํ•˜๋Š” batch_get_query_execution๊ฐ€ ์กด์žฌ
- ํ•ด๋‹น ๊ธฐ๋Šฅ์„ ์‚ฌ์šฉํ•ด์„œ Athena๊ฐ€ ์ฟผ๋ฆฌ๋ฅผ ๋Œ๊ณ  ์žˆ๋Š”์ง€ ์ƒํƒœ๋ฅผ ํ™•์ธํ•  ์ˆ˜ ์žˆ๋‹ค.

 

boto3๋ฅผ ์‚ฌ์šฉํ•ด์„œ Athena ์ฟผ๋ฆฌ ์‹คํ–‰ ์ƒํƒœ ํ™•์ธํ•˜๊ธฐ

Python ๊ธฐ๋ฐ˜์˜ AWS SDK์ธ boto3๋ฅผ ์‚ฌ์šฉํ•ด์„œ Athena ์ฟผ๋ฆฌ๋ฅผ ์‹คํ–‰ํ•  ์ˆ˜ ์žˆ๋‹ค.

ํ•˜์ง€๋งŒ boto3๋กœ Athena ์ฟผ๋ฆฌ๋ฅผ ์‹คํ–‰ํ•  ๊ฒฝ์šฐ async๋กœ ๋Œ์•„๊ฐ€๊ธฐ ๋•Œ๋ฌธ์— ์ฟผ๋ฆฌ๋ฅผ ๋‚ ๋ ธ๋‹ค๊ณ  ๋ฐ”๋กœ ๊ฒฐ๊ณผ๋ฅผ ํ™•์ธํ•  ์ˆ˜ ์žˆ๋Š” ๊ฒŒ ์•„๋‹ˆ๋‹ค. ๊ทธ๋Ÿด ๋•Œ ์‚ฌ์šฉํ•˜๊ธฐ ์œ„ํ•œ batch_get_query_execution๊ฐ€ boto3์— ์กด์žฌํ•œ๋‹ค.

 

query ์‹คํ–‰ํ–ˆ์„ ๋•Œ์˜ ExecutionID๋ฅผ ์•Œ๊ณ  ์žˆ๋‹ค๋ฉด ์•„๋ž˜์™€ ๊ฐ™์ด batch_get_query_execution์— ๋„ฃ์–ด์„œ status ๊ฐ’์„ ํ™•์ธํ•  ์ˆ˜ ์žˆ๋‹ค. ์—ฌ๋Ÿฌ๊ฐ€์ง€ ์ •๋ณด๊ฐ€ return ๋˜์ง€๋งŒ ์•„๋ž˜ ์ฝ”๋“œ ๊ธฐ์ค€์œผ๋กœ๋งŒ ๊ฐ„๋‹ค๋ฉด status ๊ฐ’ QUEUED / RUNNING / SUCCEEDED / FAILED / CANCELLED ์ค‘ ํ•˜๋‚˜๊ฐ€ return ๋˜๊ฒŒ ๋œ๋‹ค. ์ด๋ฅผ ํ™•์ธํ•˜์—ฌ ์ฟผ๋ฆฌ ์‹คํ–‰ ํ›„ ๊ฒฐ๊ณผ๋ฅผ ๋‹ค์‹œ ํ™•์ธํ•˜๊ธฐ๊นŒ์ง€ while ๊ฐ™์€ ๊ฑธ ์‚ฌ์šฉํ•ด์„œ ๋Œ€๊ธฐ๋ฅผ ๊ฑธ์–ด๋†“์„์ˆ˜๋„ ์žˆ์„ ๊ฒƒ์ด๋‹ค.

 

์˜ˆ์‹œ์˜ QueryExecutionIds๋Š” An array of query execution IDs. ์ฆ‰, array๋กœ ์—ฌ๋Ÿฌ ๊ฐ’์„ ๋„ฃ์–ด์„œ ์กฐํšŒ๋„ ๊ฐ€๋Šฅํ•˜๋‹ค. ์ตœ๋Œ€ 50๊ฐœ๊นŒ์ง€ ์กฐํšŒ๊ฐ€ ๊ฐ€๋Šฅํ•œ ๋“ฏ ํ•˜๋‹ค.

import boto3

athena = boto3.client('athena',
                   aws_access_key_id='API_KEY',
                   aws_secret_access_key='SECRET_KEY',
                   region_name='REGION'
                   )

response = athena.batch_get_query_execution(QueryExecutionIds=['QueryExecutionIds'])
status = response['QueryExecutions'][0]['Status']['State']

print(status)

#athena ๊ด€๋ จ ๋ฆฌ์„œ์น˜ ๋‚ด์—ญ:

๊ด€๋ จ๊ธ€ ๋”๋ณด๊ธฐ

๋Œ“๊ธ€ ์˜์—ญ