Earlier this year, I joined a team that had been tasked with working on some exciting green-field projects to support better flow of data within our organization. Our team took the opportunity to step into the present and adopt some more modern and feature-rich tools, such as GitHub for remote…

How to download data from Snowflake — Snowflake is a cloud data platform that offers solutions to a variety of data management problems faced by modern organizations. Snowflake is most commonly used by organizations as a data warehouse to perform online analytical processing (OLAP). In this respect, Snowflake can be used to solve the same class of…

AWS Lambda is an Amazon Web Services (AWS) compute service offering that allows developers to write code in a variety of programming languages and deploy their code on a highly available and completely managed compute infrastructure. AWS Lambda functions have become an essential component in many modern cloud applications and…

Essential concepts, patterns, and modules — Contents What is functional programming? Functions, functions, functions Immutable data types Useful modules Conclusion References Python is a general purpose, multi-paradigm programming language. That means a programmer using Python can write just about any type of program using one or more programming paradigms. Programming paradigms are a manner of programming…

Write and deploy AWS Lambda functions with more confidence — Contents Introduction Tip #1: CPU power and memory scale proportionally Tip #2: Simplicity is a virtue Tip #3: Mind your concurrency Tip #4: Use native handling of partial batch failures Tip #5: Necessary runs only Tip #6: Avoid using AWS Lambda Conclusion References Introduction AWS Lambda is a highly scalable compute service…

Visually explore, tune, and select machine learning models — Contents Introduction Classification demo Conclusion References Introduction It is well known that data is a critical part of machine learning (ML) projects — especially, the data that is used as input to ML algorithms. …

Contents Introduction Overview Configuring Time Travel Querying/cloning historical data/objects Undropping historical objects Conclusion References Introduction Snowflake is a cloud-based data warehouse similar to others such as Amazon Redshift and Google BigQuery. …

Contents Introduction Prerequisite setup Create an Amazon S3 bucket Create a storage integration Create an external stage Create a stored procedure Schedule stored procedure execution using tasks Recovery Conclusion References Introduction Snowflake offers multiple layers of native data recovery features such as Time Travel and Fail Safe, which allow querying data…

I recently found myself needing to generate subsets of collections. In one situation I needed to generate all combinations of possibly interacting diseases (up to 6 diseases from a set of ~24 diseases) to diagnose a given set of symptoms. …