Joe Zack

Jamie from https://dotnetcore.show/ and Allen, ya know, from Coding Blocks, sat down together at NDC London to talk about the hot topics from the conference as well as how to get the most out of any conference you attend. If you’re reading this episodes show notes via your podcast player, you can find this episode’s full show notes at https://www.codingblocks.net/episode126 where you can join in on the conversation. Sponsors Datadog – Sign up today at codingblocks.net/datadog for a free 14 day trial and get a free Datadog t-shirt after creating your first dashboard. Read Datadog’s new State of Serverless research report that breaks down the state of Serverless, with a detailed look at AWS Lambda usage. Educative.io – Level up your coding skills, quickly and efficiently. Visit educative.io/codingblocks to get 10% off any course or annual subscription.Clubhouse – The fast and enjoyable project management platform that breaks down silos and brings teams together to ship value, not features. Sign up to get two additional free months of Clubhouse on any paid plan by visiting clubhouse.io/codingblocks. How to get the most out of a Conference If the conference has an app – I highly recommend downloading it – typically latest breaking changes to venue, rooms, talks, etc. will be in the appAttend talks that are outside your immediate realm of knowledge – get exposed to new things, new ideas, new ways of thinkingWalk away with fewer “unknown unknowns” and gain some more “known unknowns”Provides you with things to search when you get back from the conferencePicking the talks you want to attendSometimes you have to sacrifice bigger names just to attend talks that may pique your interestKnow that what you’re seeing on stage for an hour was probably weeks worth of effort to make it go off without a hitch – so when you try to replicate these things at home, don’t lose hope when your try isn’t as smooth as what you saw on stageThis next bit goes for Meetups, Conferences, etc – Get involved in conversations – don’t just sit on the sideline – many developers are introverts, but to truly get the most out of a conference you want to have some meaningful discussionsPacman effect – leave a gap when you’re standing in a group having a conversationTake advantage of eating times – find a table with an open spot and don’t pick up your phone!!! Say good morning or good afternoon! “What’s been your favorite talk today?”When it’s “drinking time”, talk to people. If you’re not a drinker, grab a water or a soda and join in on the conversationTry and reach out BEFORE the conference online – Twitter, Facebook, Slack, etc – try and find out who all is going to be attending and try to make a point to meet up at the event! Makes things much less awkward when you’ve planned a meeting rather than just shouldering your way in.Be a wingman/wingwoman or bring one along – help introduce people to your ring of contacts’Maybe sign up to be a speaker at one of these things! If you watch the other folks giving presentations, you’ll see they’re regular people just sharing the things they’re passionate aboutThe big names in the industry became big names because they took that first step – you don’t become a big name overnight Must-see Presentations Allen Underwood (shameless plug) – Big Data Analytics in Near-Real-Time with Apache Kafka StreamsTwitter: @theallenu [https://www.twitter.com/theallenu]Summary: https://ndc-london.com/talk/big-data-analytics-in-near-real-time-with-apache-kafka-streams/Actual talk here: Coming SoonLaura Silvanavičiūtė (this was my favorite of the entire conference) – How to code music?Twitter: @laurasilvanavi [https://www.twitter.com/@laurasilvanavi]Summary: https://ndc-london.com/talk/how-to-code-music/Actual talk here: Coming SoonTess Ferrandez-Norlander – We are the Guardians of our FutureSummary: https://ndc-london.com/talk/keynote-we-are-the-guardians-of-our-future/Actual talk here: https://www.youtube.com/watch?v=2YjrmgFJ_S8Clifford Agius – 3D printed Bionic Hand a little IOT and a Xamarin Mobile AppTwitter: @CliffordAgius [https://www.twitter.com/CliffordAgius]Summary: https://ndc-london.com/talk/3d-printed-bionic-hand-a-little-iot-and-a-xamarin-mobile-app/Actual talk here: ComingSoonBlog version: https://cliffordagius.co.uk/2019/10/06/3d-printing-a-hand/Carl Franklin from .NET Rocks – Deep Dive on Server-Side BlazorTwitter: @carlfranklin [https://www.twitter.com/carlfranklin]Summary: https://ndc-london.com/talk/deep-dive-on-server-side-blazor/Actual talk here: Coming SoonDavid Fowler: SignalR Deep Dive: Building ServersTwitter: @davidfowl [https://www.twitter.com/davidfowl]Summary: https://ndc-london.com/talk/signalr-deep-dive-building-servers/Actual talk here: Coming SoonSteve GordonTwitter: @stevejgordonSummary: https://ndc-london.com/talk/turbocharged-writing-high-performance-c-and-net-code/Actual talk here: Coming SoonDavid James – Turning a side project into a business 10 lessons in 10 minutesTwitter: @davidjames [https://www.twitter.com/davidjames]Summary: https://ndc-london.com/talk/lightning-talks-9/Actual talk here: Coming SoonSteve SandersonTwitter: @stevesanderson [http://www.twitter.com/stevesanderson]Summary: https://ndc-london.com/talk/blazor-a-new-framework-for-browser-based-net-apps-1/Actual talk here: Coming Soon Notes from some of the Talks Machine Learning – we as developers need to take much more care in what we release to the worldA number of talks / discussion panels revolved around this topicEven with good intentions, you can make something that has consequences that aren’t easy to seeKnowing your data intimately is the key to everything – but, you need to have different perspectives on the data – it’d be really easy to get laser focused on what you think makes for a good set of data for a model, and miss the pieces that actually provide the best modelMicrosoft’s ethical approach to AI – AI Principleshttps://www.microsoft.com/en-us/ai/responsible-aiCities with most camera coverage? Looks like Allen got it wrong – London isn’t in the first spot anymore, but they’re still top 10!https://www.comparitech.com/vpn-privacy/the-worlds-most-surveilled-cities/ Favorite part of the conference? “Why go if you can just watch the videos?” Interacting with peopleThanking the people who make an impact on your daily life Miscellaneous It’s not “Steve Ardalis” as Allen said – it’s Steve Smith, better known as @Ardalis online!Twitter: @ardalis [https://twitter.com/ardalis] App Center – DevOps Pipeline for Mobilehttps://appcenter.ms/ Zac Braddy / Jamie Taylor plan to launch a new podcast! Jamie doesn’t care about tabs or spaces…what?!?!He also has other podcasts in the work, but they’re on hold at the moment…stay tuned!

We dive into declarative vs imperative query languages as we continue to dive into Designing Data-Intensive Applications while Allen is gallivanting around London, Michael had a bullish opinion, and Joe might not know about The Witcher. If you’re reading this episodes show notes via your podcast player, you can find this episode’s full show notes at https://www.codingblocks.net/episode125 where you can join in on the conversation. Sponsors Datadog – Sign up today at codingblocks.net/datadog for a free 14 day trial and get a free Datadog t-shirt after creating your first dashboard. Read Datadog’s new State of Serverless research report that breaks down the state of Serverless, with a detailed look at AWS Lambda usage. Educative.io – Level up your coding skills, quickly and efficiently. Visit educative.io/codingblocks to get 10% off any course or annual subscription.Clubhouse – The fast and enjoyable project management platform that breaks down silos and brings teams together to ship value, not features. Sign up to get two additional free months of Clubhouse on any paid plan by visiting clubhouse.io/codingblocks. Survey Says Anonymous VoteSign in with WordpressHow do you pronounce data?DAY-TA, where that first a is a long a ... dādəDAA-TA, where that first a is a short a ... dadəvote News We thank everyone that left us some great reviews:iTunes: 3divint, RyansWorld23Stitcher: Thomasvc, thew_s_witcher4, DaveTheShirt, Yarpskendaya Get your shin kicking shoes on and sign up for the South Florida Software Developers Conference 2020, February 29th, where Joe will be giving his talk, Streaming Architectures by Example. (fladotnet.com)Come meet us at the 15th annual Orlando Code Camp & Tech Conference, March 28th. Grab some swag and kick us in the shins. (orlandocodecamp.com) Query Languages Declarative vs Imperative The relational model introduced a declarative query language: SQL.Prior models used imperative code.An imperative language performs certain operations in a certain order, i.e. do this, then do that.With a declarative query language, you specify the pattern of data you want, the conditions that must be met, any sorting, grouping, etc.Note that you don’t specify how to retrieve the data. That is left to the optimizer to figure out.Declarative languages are attractive because they are shorter and easier to work with.Consider UI frameworks where you declaratively describe the UI without needing to write code that actually draws a button of a specific size in a specific place with a specific label, etc.Additionally, declarative languages hide the implementation details.This means it’s easier to continue using the code as the underlying engine is updated, be it a database, UI framework, etc.This also means that the declarative code can take advantage of performance enhancements with little to no change (often) to the declarative code.Because declarative languages only specify the result, instead of how to get the result, they are often more likely to be able to take advantage of parallel execution.Conversely, because imperative code needs to happen in a specific order, it’s more difficult to parallelize. MapReduce Made popular by Google, MapReduce is a programming model meant for processing large amounts of data in bulk in a horizontally distributed fashion.Some NoSQL databases, such as MongoDB and CouchDB, support MapReduce in a limited form as a way to perform read-only queries across many documents.MapReduce isn’t a declarative query language but it’s also not completely an imperative query API either.This is because to use it, you’re implementing the Template Pattern (episode 16).With MapReduce, you implement two methods: map() and reduce().The map() and reduce() functions are pure functions.They can only use the data passed into them, they can’t perform additional queries, and they must not have side effects.Pure functions are a concept used in functional programming.From a usability perspective though, it does require writing two functions that are somewhat tied to each other, which may be more effort than just writing a single SQL query.Plus a purely declarative SQL query is better able to take advantage of the optimizer.For this reason, MongoDB added a declarative query language called the aggregation pipeline to wrap the MapReduce functionality.It’s expessiveness is similar to a subset of SQL but in a JSON syntax. Graph-Like Data Models Relationships, particularly many-to-many, are an important feature for distinguishing between when to use which data model.As relationships get even more complicated, graph models start to feel more natural.Where as document databases have documents, and relational databases have tables, rows, and columns, graph databases have:Vertices: Nodes in the graphEdges: Define the relationships between nodes, and can contain data about those relationships.Examples of graph-like data:Social graphs: Vertices are the entities (people, media, articles), and edges are the relationships (friends with, likes, etc.)Web graph: Vertices are the pages, and edges are the links.Maps: Addresses are the vertices, and roads, rails, sidewalks are the edges.There are some things that are trivial to express in a graph query that are really hard any other way.For example, fetch the top 10 people that are friends with my friends, but not friends with me, and liked pages that I like sorted by the count of our common interests.These queries work just like graph algorithms, you define how the graph is traversed.Graph databases tend to be highly flexible since you can keep adding new vertices and nodes without changing any other relationships.This makes graphs great for evolvability. Resources We Like Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems by Martin Kleppmann (Amazon)Grokking the System Design Interview (Educative.io)Design Patterns Part 2 – Oh behave! (episode 16)Developer Survey Results 2019 (Stack Overflow)Graph Algorithms (episode 85)CloudSQL with Amy Krishnamohan (gcppodcast.com) Tip of the Week Is there an equivalent of tail -f on Windows? (Stack Overflow)Recursively find files whose content matches a regex pattern and display the first 10 lines for context: Get-ChildItem .\*.txt -Recurse | Select-String -Pattern 'MyPattern' -context 10,0 Use the Microsoft Application Inspector to identify and surface well-known features and other interesting characteristics of a component’s source code to determine what it is and/or what it does. (GitHub)Automatically silence those pesky, or worse: embarrassing, notifications while screensharing on your Mac. (Muzzle)

While we continue to dig into Designing Data-Intensive Applications, we take a step back to discuss data models and relationships as Michael covers all of his bases, Allen has a survey answer just for him, and Joe really didn’t get his tip from Reddit. This episode’s full show notes can be found at https://www.codingblocks.net/episode124, in case you’re reading this via your podcast player, where you can be a part of the conversation. Sponsors Datadog.com/codingblocks – Sign up today for a free 14 day trial and get a free Datadog t-shirt after creating your first dashboard. Educative.io – Level up your coding skills, quickly and efficiently. Visit educative.io/codingblocks to get 10% off any course or annual subscription.Clubhouse – The fast and enjoyable project management platform that breaks down silos and brings teams together to ship value, not features. Sign up to get two additional free months of Clubhouse on any paid plan by visiting clubhouse.io/codingblocks. Survey Says Anonymous VoteSign in with WordpressWhich keyboard do you use?CODE Keyboard*Das Keyboard*ErgoDox*Kinesis Advantage*Kinesis Freestyle*Apple Magic KeyboardApple Aluminum Wired KeyboardMicrosoft Sculpt Ergonomic KeyboardMicrosoft Surface KeyboardUltimate Hacking KeyboardOther Mechanical KeyboardOther Chiclet KeyboardOther. The other _other_ keyboard.The one that came with my laptop. Cause ain't nobody got time for carrying around fancy keyboards.vote News Thank you for the awesome reviews:iTunes: Kampfirez, Ameise776, JozacAlanOutlaw, skmetzger, Napalm684, Dingus the FirstGet your tickets now for NDC { London }, January 27th – 31st, where you can kick Allen in the shins where he will be giving his talk, Big Data Analytics in Near-Real-Time with Apache Kafka Streams. (ndc-london.com)Hurry and sign up for the South Florida Software Developers Conference 2020, February 29th, where Joe will be giving his talk, Streaming Architectures by Example. This is a great opportunity for you to try to kick him in the shins. (fladotnet.com)The CB guys will be at the 15th Annual Orlando Code Camp & Tech Conference, March 28th. Sign up for your chance to kick them all in the shins and grab some swag. (orlandocodecamp.com) Relationships … It’s complicated Normalization Relational databases are typically normalized.A quick description of normalization would be associating meaningful data with a key and then relating data by keys rather than storing all of the data together.Normalization reduces redundancy and improve data integrity.Relational normalization has several benefits:Consistent styling and spelling for meaningful values.No ambiguity, even when text values are coincidentally the same, for example, Georgia the state vs Georgia the country.Updating meaningful values is easy since there is only one spot to change.Language localization support can be easier because you can associate different meaningful values with the same key for each supported language.Search for hierarchical relationships can be easier, for example, getting a list of cities for a particular state.This can vary based on how the data is stored. See episode 28 and episode 29 for more detailed discussions related to some strategies.There are legitimate reasons for having denormalized data in a relational database, like faster searches, although there might be better tools for the specific use case. Relationships … In Document Databases Document databases struggle as relationships get more complicated.Document database designers have to make careful decisions about where data will be stored.A big benefit of document databases is locality, meaning all of the relevant data for an entity is stored in one spot. Fetching an order object is one simple get in a document database, while the relational database might end up being more than one query and will surely join multiple tables. In Relational Databases There are several benefits of relational database relationships, particularly Many-to-One and Many-to-Many relationshipsTo illustrate a Many-to-One example, there are many parts associated to one particular computer.To illustrate a Many-to-Many example, a person can be associated to many computers and a computer can be associated to many people. As your product matures, your database (typically) gets more complicated. The relational model holds up really well to these changes over time. The queries get more complicated as you add more relationships, but your flexibility remains. Query Optimization A query optimizer, a common part of popular RDBMSes, is responsible for deciding which parts of your written query to execute in which order and which indexes to use.The query optimizer has a huge impact on performance and is a big part of the reason why proprietary RDBMSes like Oracle and SQL Server are so popular.Imagine if you, the developer, had to be smarter about the order that you joined your tables and the order of items in your WHERE clause …and then ratios of data in the tables were different in production vs development,and then a new index was added, …The query optimizer uses advanced statistics about your data to make smart choices about how to execute your query.A key insight into the relational model is that the query optimizer only has to be built once and everybody benefits from it.In document databases, the developers and data model designers have to consider their designs and querying constantly. How to choose Document vs Relational Document Databases … Better performance in some use cases because of locality.Often scale very well because of the locality.Are flexible in what they can store, often called “schemaless” or “schema on read”, but put another way, this is a lack of enforced integrity.Have poor support for joining because you have to fetch the whole document for a simple lookup.Require extra care when designing because it’s difficult to change the document formats after the fact and because there is no generic query optimizer available. Relational Databases … Can provide powerful relationships, particularly with highly connected data.However, they don’t scale horizontally very well. Resources We Like Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems by Martin Kleppmann (Amazon)Grokking the System Design Interview (Educative.io)Generate metrics from your logs to view historical trends and track SLOs (Datadog)Hierarchical Data – Adjacency Lists and Nested Set Models (episode 28)Hierarchical Data cont’d – Path Enumeration and Closure Tables (episode 29) Tip of the Week Presto – The Distributed SQL Query Engine for Big Data. (prestodb.io)Use the Files app in iOS to proxy files from Box or Google Drive (support.apple.com)Pin tabs in Chrome for all of your must have open tabs. (support.google.com)Use the Microsoft Authenticator to keep all of your one-time passwords in sync across all of your devices. And it requires you authenticate with it to even see the OTPs! (App Store, Google Play)Combine Poker with learning with Varianto:25’s Git playing cards. (varianto25.com)Search your Gmail for unread old emails with queries like before:2019/01/01 is:unread.The new JetBrains Mono font is almost as awesome as the page that describes it. (JetBrains)

We’re comparing data models as we continue our deep dive into Designing Data-Intensive Applications as Coach Joe is ready to teach some basketball, Michael can’t pronounce 6NF, and Allen measured some geodesic distances just this morning. For those reading these show notes via a podcast player, this episode’s full show notes can be found at https://www.codingblocks.net/episode123 where you can also join in on the conversation. Sponsors Datadog.com/codingblocks – Sign up today for a free 14 day trial and get a free Datadog t-shirt after creating your first dashboard. Educative.io – Level up your coding skills, quickly and efficiently. Visit educative.io/codingblocks to get 20% off any course or, for a limited time, get 50% off an annual subscription.ABOUT YOU – One of the fastest growing e-commerce companies headquartered in Hamburg, Germany is always on the hunt for new motivated team members like you. Apply now at aboutyou.com/job. Survey Says Anonymous VoteSign in with WordpressWhich data model do you prefer?Relational Model. I love many to many joins. 6NF all the things.Document Model. I'll worry about the data structure when I read it.Graph Model. It just sounds cool. "Oh you're using relational data models? That's cute."Polyglot Persistence. I'll use what I think makes sense for the use case.vote News We thank everyone that took a moment to leave us a review:iTunes: BoulderDude333, the pang1, fizch26Hurry up and get your tickets now for NDC { London }, January 27th – 31st, where Allen will be giving his talk, Big Data Analytics in Near-Real-Time with Apache Kafka Streams. This is your chance to kick him in the shins on the other side of the pond. (ndc-london.com)Sign up for your chance to kick Joe in the shins at the South Florida Software Developers Conference 2020, February 29th, where he will be giving his talk, Streaming Architectures by Example. (fladotnet.com)Want a chance to kick all three Coding Blocks hosts in the shins? Sign up for the 15th Annual Orlando Code Camp & Tech Conference, March 28th, for your chance to kick them all in the shins and grab some swag. (orlandocodecamp.com) Data Models Data models are one of the most important pieces of developing software.It dictates how the software is written.And it dictates how we think about the problems we’re solving.Software is typically written by stacking layers of modeling on top of each other.We write objects and data structures to reflect the real world.These then get translated into some format that will be persisted in JSON, XML, relational tables, graph db’s, etc.The people that built the storage engine had to determine how to model the data on disk and in memory to support things like search, fast access, etc.Even further down, those bits have to be converted to electrical current, pulses of light, magnetic fields and so on.Complex applications commonly have many layers: APIs built on top of APIs.What’s the purpose of these layers? To hide the complexity of the layer below it.The abstractions allow different groups of people (potentially with completely different skillsets) to work together.There are MANY types of data models, all with different usages and needs in mind.It can take a LOT of time and effort to master just a single model.Data models have a HUGE impact on how you write your applications, so its important to choose one that makes sense for what you’re trying to accomplish. Relational Model vs Document Model Best-known model today is probably the ones based on SQL.The relational model was proposed by Edgar Codd back in 1970.The relational model organizes data into relations (i.e. tables in SQL) where each relation contains an unordered collection of tuples (i.e. rows in SQL).People originally doubted it would work but it’s dominance has lasted since the mid-80’s, which the author points out is basically an eternity in software.Origins were based in business data processing, particularly transaction processing.There have been a number of competing data storage and querying approaches over the years.Network and Hierarchical models in 70’s and 80’s,Object databases were competitors in the late 80’s and early 90’s,XML databases,Basically a number a competitors over the years but nobody has dethroned the relational database.Almost everything you see and use today has some sort of relational database working behind it. NoSQL NoSQL is the latest competitor to Relational Databases.It was originally intended as a catchy Twitter hashtag for a meetup about open source, distributed, non-relational databases.It has since been re-termed to “Not only SQL”.What needs does NoSQL aim to address?The need for greater scalability than traditional RDBMS’s can typically achieve, including very large datasets and fast writes.The desire for FOSS (free and open source software), as opposed to very expensive, commercial RDBMS’s.Specialized query operations that are not supported well in the relational model.Shortcomings of relational models – need for more dynamic and/or expressive data models.Different applications (or even different pieces of the same application) have different needs and may require different data models. For that reason, it’s very likely that NoSQL won’t replace SQL, but rather it’ll augment it.This is referred to as polyglot persistence. Object-Relational Mismatch Most applications today are written in an object oriented programming language.There’s typically a translation layer required to map the relational data models to an object model.The disconnect between models can be referred to as impedance mismatch.Frameworks like ActiveRecord, Hibernate, Entity Framework, etc., can reduce the boilerplate code needed for the translation but typically don’t fully hide the impedance mismatch issues. Resources We Like Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems by Martin Kleppmann (Amazon)Grokking the System Design Interview (Educative.io)Monitor Azure DevOps workflows and pipelines with Datadog (Datadog)Monitor Amazon EKS on AWS Fargate with Datadog (Datadog)Best practices for tagging your infrastructure and applications (Datadog)Introducing: Educative Subscriptions (Educative.io)Santosh Hari – Not all data is created equal: NoSQL (YouTube)TIOBE Index (tiobe.com) Database Schema for Multiple Types of Products Tip of the Week Got data? Use DataGrip. One tool for many databases. (JetBrains)KafkaHQ – A Kafka GUI for topics, data, consumer groups, schema registry and more. (GitHub)Grafka – A GraphQL interface for Apache Kafka (GitHub)Use Google Maps to measure geodesic distances (citylab.com)How to undo (almost) anything with Git (GitHub)Will Save the Galaxy for Food by Yahtzee Croshaw (Amazon)

We dig into what it takes to make a maintainable application as we continue to learn from Designing Data-Intensive Applications, as Allen is a big fan of baby Yoda, Michael’s index isn’t corrupt, and Joe has some latency issues. In case you’re reading this via your podcast player, this episode’s full show notes can be found at https://www.codingblocks.net/episode122 where you can join in on the conversation. Sponsors Educative.io – Level up your coding skills, quickly and efficiently. Visit educative.io/codingblocks to get 20% off any course or, for a limited time, get 50% off an annual subscription.ABOUT YOU – One of the fastest growing e-commerce companies headquartered in Hamburg, Germany is always on the hunt for new motivated team members like you. Apply now at aboutyou.com/job. Survey Says Anonymous VoteSign in with WordpressWhich sci-fi series is best?Star Trek. Dammit Jim, I'm a doctor, not a doc ... ok, fine.Star Wars. Han shot first.vote News Thank you for taking a moment out of your day to leave us a review:iTunes: Kodestar, CrouchingProbeHiddenCannon Stitcher: programticalpragrammerer, Patricio Page, TheOtherOtherJZ, Luke GarriganBe careful about sharing/using code to/from Stack Overflow! You need to be aware of the licensing and what it might mean for your application.What is the license for the content I post? (Stack Overflow)Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) (Creative Commons)Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) (tldrlegal.com)We Still Don’t Understand Open Source Licensing (episode 5)Get your tickets now for NDC { London } for your chance to kick Allen in the shins where he will be giving his talk Big Data Analytics in Near-Real-Time with Apache Kafka Streams. (ndc-london.com) Maintainability A majority of cost in software is maintaining it, not creating it in the first place.Coincidentally most people dislike working on these legacy systems possibly due to one or more of the following:Bad code,Outdated platforms, and/orMade to do things the system wasn’t designed to do.We SHOULD build applications to minimize the pain during the maintenance phase which involves using the following design principals:Operability – make it easy to keep the system running smoothly.Simplicity – make it easy for new developers to pick up and understand what was created, remove complexity from the system.Evolvability – make it easy for developers to extend, modify, and enhance the system. Operability Good operations can overcome bad software, but great software cannot overcome bad operations.Operations teams are vital for making software run properly.Responsibilities include:Monitoring and restoring service if the system goes into a bad state.Tracking down the problems.Keeping the system updated and patched.Keeping track of how systems impact each other.Anticipating and planning for future problems, such as scale and/or capacity.Establish good practices for deployments, configuration management, etc.Doing complicated maintenance tasks, such as migrating from one platform to another.Maintaining security.Making processes and operations predictable for stability.Keeping knowledge of systems in the business even as people come and go. No tribal knowledge.The whole point of this is to make mundane tasks easy allowing the operations teams to focus on higher value activities.This is where data systems come in.Get visibility into running systems, i.e. monitoring.Supporting automation and integration.Avoiding dependencies on specific machines.Documenting easy to understand operational models, for example, if you do this action, then this will happen.Giving good default behavior with the ability to override settings.Self-healing when possible but also manually controllable.Predictable behavior. Simplicity As projects grow, they tend to get much more complicated over time.This slows down development as it takes longer to make changes.If you’re not careful, it can become a big ball of mud.Indicators of complexity:Explosion of state space,Tight coupling,Spaghetti of dependencies,Inconsistent coding standards such as naming and terminology,Hacking in performance improvements,Code to handle one off edge cases sprinkled throughout.Greater risk of introducing bugs.Reducing complexity improves maintainability of software.For this reason alone, we should strive to make our systems simpler to understand.Reducing complexity DOES NOT mean removing or reducing functionality. Accidental complexity – Complexity that is accidental if it is not inherent in the problem that the software solves, but arises only from the implementation.Ben Moseley and Peter Marks, Out of the Tar Pit How to remove accidental complexity? AbstractionAllows you to hide implementation details behind a facade.Also allows you to reduce duplication as abstractions can allow for reuse among many implementations.Examples of good abstractions:Programming languages abstract system level architectures such as CPU, RAM registers, etc.SQL is an abstraction over complex memory and data structures.Finding good abstractions is VERY HARD.This has become apparent with distributed systems as these abstractions are still being explored. Evolvability It’s VERY likely that your system will need to undergo changes as time moves on.Agile methods allow you to adapt to change. Some methods include:Test Driven Development RefactoringHow easy you can modify a data system is closely linked to how simple the system is.Rather than calling it agility, the book refers to it as evolvability. Resources We Like Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems by Martin Kleppmann (Amazon)Grokking the System Design Interview (Educative.io)All permutations of English letters with a handy search feature (libraryofbabel.info) What is the license for the content I post? (Stack Overflow)Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) (Creative Commons)Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) (tldrlegal.com)We Still Don’t Understand Open Source Licensing (episode 5)Streaming process output to a browser, with SignalR and C# (Coding Blocks, dev.to)Googlewhack – A Google query using two words, without quotes, that returns a single search result. (Wikipedia)The Website is Down – Sales Guy vs. Web Dude (YouTube – Explicit)Software Design Anti-patterns (episode 65)Clean Architecture – How to Quantify Component Coupling (episode 72)Static Analysis w/ NDepends – How good is your code? (episode 15)Static Analysis of Open Source .NET Projects (Coding Blocks)Does Facebook still use PHP? (Quora) Tip of the Week Need to move data from here to there? Check out Fluentd. (fluentd.org)Use NUnit’s Assert.Multiple to assert that multiple conditions are all met. (GitHub)Turns out C#’s where generic type constraints are even cooler than you might have realized. (docs.microsoft.com)Best Practices for Response Times and Latency (GitHub)