Greetings !
We have a log table where we maintain daily rowcount of all major tables in snowflake.
To monitor data completeness/duplication , i need to send an email when rowcount of a particular table falls out of its min/max pre-defined threshold value.
I know i will have to use python logic for this but unsure about how exactly i query the snowflake using python and get output as true/false so that i run following email component based on that.
Thanks.
Hi @Ajinkya, there is a pattern that you can use that should not require a Python script to make the true/false determination you are looking for. You would want to use Query to Scalar component. The idea is that you have a job variable that will hold the true or false result. You then write a query against your log table using a CASE statement to determine if the the row count is outside your min/max values and then based on that you would return a true/false or 0/1, etc. This get's stored into your Job variable. The component after the Query to Scalar would be a "If" flow where you would use the Job variable to determine the path. Based on the path you can choose to send an email or not.
I hope this helps give you some context on how this scenario could be handled.
