
An active Player in the game of technology


Brian Baliach


Last week, I attended an AI conference that showcased groundbreaking innovations in Kenya’s North Horr Constituency—a region demonstrating how AI and digital tools can revolutionize communities, even in the most challenging environments.

I had the pleasure of sharing the stage as part of a panel discussion, where we tackled key issues such as Ethics in AI and the risks of overregulation stifling growth. It was an honor to participate alongside respected individuals, including my former lecturer, [Njeri Ngaruiya Ng'ang'a (PhD)](https://www.linkedin.com/in/ngaruiyanjeri/), who also graced the panel with her insights.

As the CTO at farmbetter, I’m constantly inspired by how technology can empower smallholder farmers to build climate resilience and thrive in the face of adversity.

### Here’s what I learned and why it matters

North Horr, home to the Chalbi Desert and the iconic Koobi Fora archaeological site, is a testament to the power of innovation. Here’s how AI is driving change:

**1\. AI for Climate Resilience:**    
Early warning systems for droughts and floods are helping farmers prepare for extreme weather.

AI-driven soil testing and water mapping are enabling pastoralists to make data-driven decisions about grazing and water sources.

**2\. Majik Water:**  
This initiative uses AI to monitor humidity and temperature, harvesting water from the air to provide clean drinking water for communities.

It's funny how many Kenyans, myself included, hadn't heard of Majik Water. I guess we were all too busy scrolling through Twitter for the latest "maandamano" updates and political drama to notice our local innovations\!

Meanwhile, this project is getting international accolades with a Netflix documentary and mentions in the Washington Post. Now, they’re scaling this technology to manage boreholes—a potential game-changer for arid regions.

**3\. EdTech:**    
Solar-powered tablets (with offline content, which means content is already downloaded on-device) are bringing education to nomadic communities, ensuring children can learn even as families move. Imagine applying this to farmer education—empowering smallholders with on-the-go access to agronomic advice.

### Key Learnings from the AI Conference

The conference reinforced how Africa can lead the AI revolution, especially  if we address critical gaps, i.e:

**1\. AI Infrastructure Gaps:**    
Reliable connectivity and access to digital tools are foundational. Without them, even the best AI solutions can’t reach those who need them most.

**2\. Localized Solutions are Key:**    
AI models must incorporate local variables and contexts. What works in one region may not work in another.

**3\. Data as a Renewable Resource:**    
We need to become better stewards of data, ensuring it’s reusable, shareable, and stored in accessible formats. Open data platforms can unlock immense value for farmers and innovators alike.

**4\. Collaboration is Key:**    
Public-private partnerships, academia, and startups must work together to scale impact. I fondly recall a mantra from my former lecturer, Mrs. Ngaruiya: "African solutions by Africa." Achieving this vision hinges on strong collaboration.

### What This Means for farmbetter 
At farmbetter, we're embracing the transformative power of AI and technology to deliver personalized, digital advice to smallholder farmers right through their mobile phones. The groundbreaking innovations in North Horr highlight the potential of homegrown solutions to address local challenges across the continent.

This inspires us to think bigger and push for uniquely African solutions like expanding early warning systems and leveraging open data.

### Closing Thoughts  
The future of agriculture in Africa lies in innovation, collaboration, and localization.

Thank you to [JKUAT](https://www.jkuat.ac.ke/) \+ [JHUB](https://jhubafrica.com/) for curating the AI conference, a collaborative space between Academia and key Industry players for AI Innovation in Africa.

I'd also like to give a shoutout to my former lecturer, [Njeri Ngaruiya Ng'ang'a (PhD)](https://www.linkedin.com/in/ngaruiyanjeri/), whose guidance and inspiration during my time on campus played a crucial role in shaping my career path and passion for leveraging technology to drive change.  
Together, we can turn challenges into opportunities and build a more resilient future for Africa’s communities.    


Tech and AI Innovation: How AI is Transforming Communities in Kenya


Transitioning from a senior software engineer to building and leading an internal team at Farmbetter marks a significant shift not only in my professional journey but also in the way I communicate my experiences. Unlike my usual technical software engineering blogs, this post delves into the broader scope of leadership, communication, and personal growth. 

Moving from a focus on code to leading a diverse team has expanded my perspective and sharpened my skills. Here, I share insights from this transformative journey, emphasizing the critical areas of communication and leadership that have been pivotal in navigating this new role.

### The Art of Over-communication

In a diverse environment with team members spread across various countries, efficient communication emerges as a key growth area. Over-communication has become not just a best practice but a necessity. Ensuring everyone is on the same page regardless of time zones enhances cooperation and prevents misunderstandings. 

Regular updates, setting clear expectations, and creating open channels for feedback are crucial. These lessons were particularly evident during my involvement in the #AgriPath project.

### The Importance of Mentorship

Navigating the complexities of corporate culture and managing human relationships can be challenging, especially when transitioning into a leadership role. Having a mentor has been invaluable in guiding me through these nuances. A mentor provides insights and advice, helping to navigate difficult situations and offering perspectives that I might not have considered. They serve as a sounding board for ideas and challenges, helping to refine leadership skills and improve interpersonal interactions. 

Having someone to share their experiences and wisdom makes the process less daunting and more structured, ultimately contributing to personal and professional growth. 
For anyone stepping into a new role or looking to excel in their current position, seeking a mentor can be a game-changer in understanding and thriving within a corporate environment.

### Embracing New Challenges

One of the most pivotal moments in my journey with Farmbetter came during the AgriPath workshop, where I faced the daunting task of presenting to high-ranking representatives. At that time, the technological aspects of the project, which Farmbetter leads, were in a challenging state, and our partners were understandably disappointed. Having just joined the team as the lead of tech, I was thrust into a situation that required not only a technical turnaround but also a strong reassurance to our partners.

This experience pushed me far out of my comfort zone, compelling me to hone my public speaking skills and become adept at handling pressure. While the static from the microphone might have been a re-occurring nemesis every time I spoke, it also taught me the importance of perseverance and adaptability in stressful situations. This challenge not only promoted my growth as a communicator but also strengthened my ability to engage meaningfully with diverse stakeholders.

The experience proved that embracing new challenges, even uncomfortable ones, is essential for personal development. Today, I'm not only a better speaker but also more confident in my ability to lead through adversity — static and all :-).

### Looking Forward

In summary, this transition has taught me that effective leadership lies in good communication, empathy, and the ability to adapt. It's the human connections and shared goals that truly drive a project's success. For those considering a similar leap, remember that growth often lies beyond the comfort zone. 

A side note: Interested in diving deeper into our work and seeing some of our research outcomes, do check out [AgriPath's website](https://www.agripath.net/) and [Farmbetter's website](https://www.farmbetter.io/). The journey towards impactful agricultural development is one worth following!

Stay Curious and 'Lettuce' Grow Together!


A Journey of Learning: From Software Engineer to Team Lead


So, What Are TypeScript Decorators?

In the simplest of terms, a decorator in TypeScript is like a wrapping paper. You take a function or a class and wrap it with another function that adds some extra features.

Decorators provide a way to add both meta-programming annotations and a declarative syntax for applying those annotations, making TypeScript more powerful and expressive. For instance, if you want a function that performs image compression and can be reused across various other functions, you can create an `ImageCompression` decorator. This decorator will wrap whichever function it is applied to, adding image compression capabilities.

To really get a grasp of this, consider an example in managing roles and permissions.

(**as Billy Butcher**) So for the deep-dive then, pay attention, because this is where it gets bloody interesting, mate.

### Let's Get Technical
### Defining a Permission Decorator

Here’s a simple example: a decorator that takes in a permission string and checks if the user has that specific permission. If the user lacks the required permission, the function returns with an unauthorized error. Note that we expect the class containing the calling function (the function what will be wrapped with this decorator) to have both a `getLoggedInUser` and `handleUnauthorizedUser` method (which is public since we can't decorate private properties) since implementations of both these methods may defer from architecture to architecture.

It is also worth noting that in this example, our decorator first runs (since it wraps the calling function) then depending on whether the user has permissions, either returns the call to `this.handleUnauthorizedUser` or the call to the calling function itself. So basically two steps, first our decorator function runs, then it returns the result of the calling function if the permission check passes.

```typescript
export interface Caller {
  getLoggedInUser: () => Promise<{ permissions: Array<{ id: string }> }>;
  handleUnauthorizedUser: () => any;
}

export type Union<T> = Caller & T;

export function requiresPermission(permission: string) {
  function actualDecorator<This>(
    target: (this: This, ...args: any) => any,
    context: ClassMethodDecoratorContext<Union<This>, any>
  ) {
    const methodName = context.name;
    if (context.private) {
      throw new Error(
        `'requiresPermission' cannot decorate private properties like ${
          methodName as string
        }.`
      );
    }

    // Bind the method to the context
    context.addInitializer(function () {
      this[methodName] = this[methodName].bind(this);
    });

    async function replacementMethod(
      this: Union<This>,
      ...args: any
    ): Promise<any> {
      const user = await this.getLoggedInUser();
      let hasPermission = false;

      try {
        user?.permissions?.forEach((p) => {
          if (p?.id === permission) {
            hasPermission = true;
            throw Error("Match found.");
          }
        });
      } catch (e) {
        // Intended to break the loop
      }

      if (!hasPermission) {
        return this.handleUnauthorizedUser();
      }
      return target.call(this, ...args);
    }

    return replacementMethod;
  }

  return actualDecorator;
}
```

### Implementing the Decorator

We’ll use the `requiresPermission` decorator in a class to manage function calls to a REST API that manages roles.

```typescript
class RESTInternalInternalRoleDataSource {

    public handleUnauthorizedUser(){
        return false; // Handle single source of unauthorized user
    }

    public getLoggedInUser() {
        return {
            id: 1,
            name: "Test User",
            permissions: [
                {
                    id: "Manage Roles",
                    name: "Manage Roles"
                },
                {
                    id: "Manage Permissions",
                    name: "Manage Permissions"
                }
            ]
        }
    }

    @requiresPermission("Manage Permissions")
    getInternalPermissions(): Array<{id: string; name: string}> {
        return [
            { id: "Manage Permissions", name: "Manage Permissions" },
            { id: "Create Permissions", name: "Create Permissions" },
            { id: "Delete Permissions", name: "Delete Permissions" }
        ];
    }

    @requiresPermission("Manage Roles")
    getInternalRoles(): Array<{id: string; name: string}> {
        return [
            { id: "1", name: "Super Admin" },
            { id: "2", name: "Admin" },
            { id: "3", name: "Finance" }
        ];
    }

    @requiresPermission("Delete Roles")
    deleteInternalRole(): boolean {
        return true; // Handle role deletion
    }
}

export default RESTInternalInternalRoleDataSource;
```

Here, each method checks the user's permissions before proceeding with its logic using the `requiresPermission` decorator. If the user doesn't have the required permission, the method handles it appropriately.

That's basically it in using Stage 3 decorators. Here's a link to [their documentation](https://devblogs.microsoft.com/typescript/announcing-typescript-5-0/#decorators) if you'd like an even deeper dive into stage 3 decorators.

For the next section, I'll be talking about how to integrate Stage 3 decorators with Next.js (App Router) since as of version 14.2.4 - currently the latest as I write this blog - doesn't offer support for Stage 3 decorators out of the box.

### The Next.js Side of Things: Adding Support for Stage 3 Decorators

Next.js v14.2.4 is a robust framework with a massive community, but unfortunately, doesn’t support TypeScript Stage 3 decorators out of the box just yet. Luckily, there's a workaround. This workaround however, isn't for the faint-hearted as it involves patching Next.js' source code. 
Don't worry though as I'll try to explain the process as clearly as I can, but feel free to comment if you still have questions!

### Setting Up with `pnpm`

First things first. You'll have to ditch `yarn` (or maybe don't if Yarn supports patching of npm packages, but you'll most definitely need to ditch `npm`.). Then download `pnpm` an efficient package manager (which also doesn't duplicate your node_modules folders by providing a central repo locally, so no more running out of storage). We’ll use `pnpm` to patch our Next.js setup.

#### Step 1: Install `pnpm`

First, let’s install `pnpm`. You can install it via npm:

```sh
npm install -g pnpm
```

Or via homebrew:

```sh
brew install pnpm
```

For more installation options, you can refer to the [official documentation](https://pnpm.io/installation).

#### Step 2: Patch Next.js

Next, patch your Next.js setup/project to support decorators. Navigate to your Next.js project directory and run:

```sh
pnpm patch next@14.2.4
```
Remember to edit the version number (the part after @) with the version you're currently using.
To check nextjs version, run:
```sh
next --version
```
pnpm will create a copy of next and output the location of this copy inside your terminal. The output should look roughly as follows:

```sh
Patch: You can now edit the package at:

  /private/var/folders/lh/jm0gx9n97ljf1hr66rbxclyh0000gn/T/1e5dc394b869fc2b77f7e91e57448382 (file:///private/var/folders/lh/jm0gx9n97ljf1hr66rbxclyh0000gn/T/1e5dc394b869fc2b77f7e91e57448382)

To commit your changes, run:

  pnpm patch-commit '/private/var/folders/lh/jm0gx9n97ljf1hr66rbxclyh0000gn/T/1e5dc394b869fc2b77f7e91e57448382'
```
Take not of the `patch-commit` command as you'll use it in the next steps.

Open the project in your favourite IDE (I prefer `vi` cause I'm 10X 😎) and edit the following file:
`dist/build/swc/options.js`

Find these entries (or similar ones):
`enableDecorators` - set this to true,
`decoratorVersion` - set this to '2022-03',
`legacyDecorator` - set this to false,

Finally, go back to your Next.js project and run patch-commit (you should have this when you ran pnpm patch):
```sh
pnpm patch-commit '/your/folder/with/next.js/copy/that/you/edited/here'
```

pnpm will add an entry automatically to your package.json that will resolve to a `patch` folder created in your project that contains the edits you just made. 

(**as Billy Butcher**) Now Go wild with decorators in your Next.js project, and maybe, just maybe, those sods will drop an update soon that sorts this mess out.

### Conclusion

Alright, that was definitely a mouthful. Let's wrap things up: You've had an introduction to TypeScript decorators, specifically Stage 3 decorators, and you've gotten a glimpse of a new package manager, `pnpm`, which you've used to skillfully patch Next.js. Now, if you're feeling adventurous, dive into the source code yourself and maybe even contribute to the Next.js repo with this patch. Maybe that long awaited update will be from you :-) The sky's the limit. 

Go on, integrate those decorators in your Next.js projects and create something truly remarkable.


TypeScript Stage 3 Decorators: A Journey Through Setup, Usage and a bonus on how to set it up in your NextJs (App Router) projects.

TypeScript Stage 3 Decorators: A Journey Through Setup and Usage


In a previous blog, [JavaScript Equality Under the Lens: Enhancing React’s Dependency Checks](https://www.balysnotes.com/javascript-equality-checks), we delved into the nuances of JavaScript data types and their equality checks. We explored values with identity and those without, examining how JavaScript stores objects by reference rather than by value. To address object comparison within React's dependency checks, we devised a method to fetch keys from an object and compare their values, though this method was limited to one level deep.

In this blog, we'll talk about a new way to do equality checks that will allow us to go n levels deep (rather than just one level) and will still be applicable not only to simple data types like numbers and strings, but also complex structures like objects and arrays. For this approach, we'll rely on an advanced but straightforward technique for value-based comparison of objects using `JSON.stringify` and `JSON.parse`.

### Revisiting Object Equality: The Challenge

JavaScript’s intrinsic method of object comparison works by comparing memory references. This means two distinct objects with identical properties and values are considered unequal:

```javascript
const objectOne = { key: "value" };
const objectTwo = { key: "value" };

console.log(objectOne === objectTwo);  // returns 'false'
```

In our previous blog, we created a function to check equality by manually comparing object keys and values. However, this function was limited to a shallow comparison:

```javascript
function shallowEqual(obj1, obj2) {
  const keys1 = Object.keys(obj1);
  const keys2 = Object.keys(obj2);

  if (keys1.length !== keys2.length) return false;

  for (let key of keys1) {
    if (obj1[key] !== obj2[key]) return false;
  }

  return true;
}
```

While this approach was somewhat effective, it fell short when dealing with nested objects and arrays. This is where `JSON.stringify` and `JSON.parse` come into play.


### The Magic of `JSON.stringify` and `JSON.parse`

In JavaScript, `JSON.stringify` converts a JavaScript object to a JSON string, while `JSON.parse` parses a JSON string to construct the JavaScript object. By converting objects into strings and comparing these string representations, we can bypass the idiosyncrasies of object reference comparison and handle nested structures efficiently:

```javascript
const objectOne = { key: "value" };
const objectTwo = { key: "value" };

console.log(JSON.stringify(objectOne) === JSON.stringify(objectTwo)); // returns 'true'
```
And at the same time still return the original object using `JSON.parse` (if there's this use case).

### Applying JSON Techniques in React

This approach is pivotal for React components' dependency checks. Let’s consider an advanced example where we use this technique in a custom hook:

```typescript
import { useCallback, useEffect, useMemo, useRef, useState } from "react";

export type ObjType = {
  [key: string | number | symbol]: any;
};

/**
 * This hook checks for equality even for nested objects or arrays. Only causes a re-render if the [obj] param changes.
 */
const useObjectEqualityChecker = (obj: ObjType) => {
  const renders = useRef(0);
  if (renders.current >= 10) {
    throw new Error(
      `Total Renders: ${renders.current}. A tad bit too much ey?`
    );
  }
  const _internalStringify = useCallback((o: any) => {
    return JSON.stringify(o);
  }, []);

  const _objString = _internalStringify(obj);
  const currentObjRef = useRef(_internalStringify(obj));
  const [currentObj, setCurrentObj] = useState<string>(_internalStringify(obj));

  useEffect(() => {
    if (_objString) {
      try {
        if (currentObjRef.current !== _objString) {
          renders.current += 1;
          currentObjRef.current = _objString;
          setCurrentObj(_objString);
        }
      } catch (e) {
        // left blank intentionally. No need for any re-render.
      }
    }
  }, [_objString]);

  return useMemo(() => {
    return JSON.parse(currentObj);
  }, [currentObj]);
};

export default useObjectEqualityChecker;
```

This hook can come in handy when building components that take arrays or objects as props, where a developer might have forgotten to use `useCallback` or `useMemo` therefore returning different memory references every render.

### Why This Approach is Better

1. **Value-Based Comparison**:
  - Unlike reference-based comparison, the JSON-based approach guarantees that objects with the same properties and values will be considered equal, thereby avoiding unexpected re-renders in React.

2. **Depth Coverage**:
  - This method seamlessly handles nested objects and arrays, ensuring accurate and deep comparisons without the need for custom deep equality functions.

3. **Simplicity and Clarity**:
  - The technique is straightforward to implement, making your code more readable and less error-prone. It reduces the cognitive load for developers trying to understand object equality logic.

4. **Consistency**:
  - Standardized behavior of `JSON.stringify` ensures consistency across different environments and usages, avoiding potential pitfalls of manually deep-checking objects or relying on third-party libraries.

### Performance Implications

Though the JSON approach is powerful, it comes with performance trade-offs, especially for larger objects or frequent comparisons:

1. **Serialization Overhead**:
  - Serializing and deserializing objects involve significant overhead compared to reference-based comparisons. For large and complex data structures, this could lead to performance bottlenecks.

2. **Frequency of Operation**:
  - If used excessively in performance-critical sections of your application, the cumulative impact might be noticeable. However, for infrequent or isolated checks, the performance impact might be negligible.

```javascript
const largeObject1 = { /* a large complex structure */ };
const largeObject2 = { /* a similarly large complex structure */ };

// Comparing using reference
console.time('Reference Comparison');
console.log(largeObject1 === largeObject2);
console.timeEnd('Reference Comparison');

// Comparing using JSON
console.time('JSON Comparison');
console.log(JSON.stringify(largeObject1) === JSON.stringify(largeObject2));
console.timeEnd('JSON Comparison');
```

In most cases, the reference comparison will be significantly faster.

### Conclusion

No one-size-fits-all solution exists in software development. Understanding the trade-offs and careful application of these techniques will greatly enhance your ability to manage and optimize equality checks in JavaScript. This method of using `JSON.stringify` and `JSON.parse` for object comparison, especially in React, guarantees accurate value-based comparisons, making your components more reliable and efficient.

Until next time, happy coding!


JavaScript Equality Under the Lens: Enhancing React’s Dependency Checks - Part 2


### Using Private GitHub/Gitlab Repos in Your Next.js Projects

Imagine this scenario: you're working on a super cool Next.js project, and you have some shared code that you'd like to use across multiple projects. You don't really want to make it public by publishing it to npm. You also don't want to pay for an npm account to make your repos private. Is there a way to keep it private, secure, yet reusable? You bet!

In the world of Next.js v13.4 and beyond, using private GitHub or GitLab repositories as npm packages is not only possible, it's also quite straightforward. The steps vary for older versions of Next.js before v13.4 (those using Page Router) and won't be covered in this blog. 

Essentially, we’ll be taking code from a private GitHub or GitLab repository and using it directly in our Next.js project. This method is perfect when you have shared components, utilities, or any other code that you need to reuse across several projects, but don't want to expose to the world (or persevere through the overhead of setting up an npm account).

Today, I'll walk you through the steps to make this happen. Buckle up, because we're about to make your life a whole lot easier!


#### Step 1: Create Your Private GitHub Repo

This is where your shared code will live. If you already have one, fantastic. If not, go ahead and create a new private repository on GitHub or Gitlab.


#### Step 2: Generate a GitHub Access Token

1. Navigate to your GitHub settings (or Gitlab settings for those using Gitlab).
2. Create a new personal access token (make sure it only has read access).
3. Copy this token and keep it safe (you’ll need it soon).


#### Step 3: Set Up Your Shared Repo

In your shared repository's `package.json`, set the `name` field to whatever you'd like to call your package, e.g., `"package-name": "1.0.0"`.


#### Step 4: Update Your Next.js Project’s `package.json`

In the `package.json` of the Next.js project where you want to use your shared code:
```json
{
  "dependencies": {
    "package-name": "git+https://<github_token>:x-oauth-basic@github.com/<user>/<repo>.git#branch-name"
  }
}
```
Replace `<github_token>`, `<user>`, `<repo>`, and `branch-name` with the appropriate values from your setup.


#### Final Steps: Configuration in Next.js

1. In your `next.config.js`, add the following:
   ```javascript
   module.exports = {
       transpilePackages: ["package-name"],
   };
   ```
   This tells Next.js to also compile your typescript code from the shared repo and transform it to javascript - that's basically what transpile means; a combination of the word 'transform' and 'compile'. Without this, you'll get module parsing errors for your typescript (.ts) files.

2. If you're using Tailwind CSS, update your `tailwind.config.js` to include your shared package:
   ```javascript
   module.exports = {
       content: [
           "./node_modules/<package_name>/src/**/*.{js,ts,jsx,tsx,html,mdx}",
           "./node_modules/<package_name>/app/**/*.{js,ts,jsx,tsx,html,mdx}",
       ],
       //… other tailwind config
   };
   ```
   Replace `<package_name>` with the name of your shared repo. Also, if you aren't using the `src` folder, feel free to adjust the values according to your setup

3. Now run `npm install` and voila! your shared package has now been installed as an NPM dependency and you can use it in your project.

#### Before We Finish Just Yet
Always remember to update the version number in your package.json with a higher value each time you make a change to your shared repo. After words, run `npm update <package_name>` where `<package_name>` is the name of your shared package/repo. This will ensure that your changes in the shared package's changes are published to your project.


### Conclusion

Incorporating private GitHub repos as npm packages in your Next.js projects not only keeps your code organized and secure but also significantly boosts your productivity. You no longer need to duplicate code or compromise your privacy by publishing sensitive code to public registries.

So, what are you waiting for? Dive in and streamline your projects today. And remember, it’s these smart shortcuts that often make a big difference in the efficiency and success of your development efforts.


Streamlining Your Next.js Project with Private GitHub/Gitlab Repositories as NPM Packages


Recently, I was asked to present on Domain Driven Design. As I delved deeper into the research, it became clear that DDD is a crucial aspect of system architecture that many developers don’t fully grasp.

Building software that aligns with complex business needs can be daunting, especially as systems grow in complexity. Domain Driven Design (DDD) offers a methodology that addresses this challenge by focusing on the business domain and its logic, thereby enabling developers to create systems that fulfill real-world business requirements.

#### What is Domain Driven Design?

**Domain Driven Design** is a software development approach that focuses on the core business domain and its logic over technological aspects. It involves creating a semantic model that mirrors real-world scenarios. This approach ensures that software components are named in a way that reflects how domain experts and stakeholders discuss the business.

Imagine a fintech company developing a loan management system. By adopting DDD, the development team creates classes like `LoanApplication`, `LoanApproval`, and `RepaymentPlan`. This makes the software model understandable to financial experts within the company, facilitating better collaboration and reducing miscommunication.

DDD therefore enhances the effectiveness of software development by aligning it more closely with business needs. By concentrating on the business’s core processes and rules, DDD ensures that the developed software is more adaptable to changing business requirements and continues to solve relevant problems.


Remember, Domain Driven Design isn't just about complex coding solutions; it's about fostering a collaborative environment where software directly solves business problems. An e-commerce platform may define a `Customer` and a `Guest User` and understanding the domain well means recognizing the difference between a `Customer` and a `Guest User`. Naming conventions like these ensure that all team members are on the same page, ultimately leading to a more cohesive and functional system.

#### Key Principles of Domain Driven Design
For this section, I'll attempt to use real world examples to better support my explanations.

1. **Ubiquitous Language**:
   Ubiquitous Language is a common language crafted by the team, including developers and domain experts. It ensures that all terms and phrases are clearly understood in the business context, eliminating ambiguities.

   **Real-World Example**: In a logistics company, terms like `Carrier`, `Shipment`, and `Delivery` are clearly defined. When developers and logistics experts use the same terminology, implementing a feature like `ShipmentTracking` becomes straightforward and error-free.

2. **Model-Driven Design**:
   This principle involves creating a conceptual model outlining all key elements of the domain, which acts as a blueprint. The model evolves over time, being continually refined to better match the domain complexities. For instance, if your domain includes entities like 'Buyer' and 'Farmer', you need to meticulously define their attributes and interactions, such as an 'Admin' entity that can 'Create User', 'Create Cash Credit Product', or 'Send Bulk SMS'.

   **Real-World Example**: In a healthcare application, entities such as `Patient`, `Doctor`, `Appointment`, and `Prescription` are modeled.A `Patient` creates an `Appointment`.  A `Doctor` treats a `Patient`. A `Patient` is given a `Prescription`.

3. **Context Boundaries (Bounded Context)**:
   Bounded Context defines clear boundaries around a particular domain model or subsystem, enabling teams to work independently without interfering with each other. For instance, in the given domain, only Admins can create cash credit products, and while Admins can create Buyers, Buyers can also register or onboard themselves.

   **Real-World Example**: In a banking application, separate bounded contexts for `Account Management` and `Loan Processing` ensure that teams working on these features do not step on each other's toes. Account management can evolve independently from loan processing.

#### Core Building Blocks of DDD

1. **Entities**:
   Entities are objects with a distinct identity that persists over time. An example is a Farmer named 'Melvin', who has a unique ID within the system.

2. **Value Objects**:
   Value Objects are defined by their attributes and are immutable. For example, a Farmer can have attributes like 'Name' and 'ID', which are value objects.

3. **Aggregates**:
   An Aggregate is a cluster of entities and value objects treated as a single unit for data changes. Each aggregate has a root entity controlling access. For example, a Farmer can be an aggregate root and may include other entities like 'Cash Credit'.

4. **Domain Events**:
   These are significant business events that trigger changes within the domain, such as placing an order or initiating bulk payments. They help decouple systems and model real-world operations.

5. **Repositories**:
   Repositories are responsible for the persistence and retrieval of aggregates, abstracting the inner workings of data access mechanisms.

#### Common Challenges in Implementing DDD

1. **Complexity in Execution**:
   Implementing DDD can seem overwhelming due to its deep modeling and upfront design focusing on domain intricacies. Starting with a critical subdomain and gradually expanding can help manage this complexity. As an example, a business building a Health Management System may start off first with patient onboarding, before moving to other sections like doctor onboarding.

2. **Integration with Non-DDD Systems**:
   Non-DDD systems might behave differently, posing challenges in maintaining data and behavior consistency. Using anti-corruption layers or dedicated interface contexts can mitigate these issues. An e-commerce platform might use API gateways to bridge legacy inventory systems with the new DDD-based application.

3. **Under- or Over-Modeling**:
   There is a risk of either over-complicating or oversimplifying the model, missing essential complexities. Regular reviews with domain experts and iterative refinement help maintain balance.

4. **Adoption and Cultural Shifts**:
   DDD requires a cultural shift towards collaboration and a strong focus on the domain. Investing in training and workshops can enhance understanding and facilitate this transition.

#### Best Practices for Domain Driven Design

1. **Collaboration Between Domain Experts and Developers**:
   Deep collaboration between business experts and developers is essential. Regular interactive sessions help refine the domain model effectively.

2. **Continuous Learning and Context Evolution**:
   The business domain evolves over time; thus, the domain model should too. Establish feedback mechanisms like retrospective meetings and model storming sessions.

3. **Keep the Model and Implementation Aligned**:
   Changes in the domain model must be reflected in the implementation. Using domain and integration events can ensure consistency.

4. **Iterative Refinement**:
   The domain model should be a living document, continuously refined based on real-world feedback and testing.

5. **Explicitly Define Context Boundaries**:
   Clear boundaries help prevent inappropriate coupling and complexity. Well-defined APIs, schemas, and contracts can manage interactions between contexts.

#### Tools and Frameworks for DDD

Technology plays a crucial role in implementing DDD effectively. Popular frameworks like NextJS, NextJS, .Net and Java Spring help build complex software systems, while modeling tools like UML diagrams and class diagrams help visualize complex domain models. For practical application, projects like Digifarm (where I'm currently employed, and the birth of this blog post) can illustrate these concepts in action.

#### Conclusion and Q&A

Effectively implementing Domain Driven Design involves more than just a technical shift; it requires a cultural transformation within development teams. This transformation fosters closer collaboration between domain experts and developers, ensuring that the software built truly serves the business objectives. By embracing DDD, organizations can create adaptable, business-aligned software capable of evolving as business needs change.

If you have any questions about DDD, feel free to ask! Whether you're curious about specific principles or need advice on tackling common challenges, I'm here to help. Stay curious!


Introduction to Domain Driven Design: Bridging the Gap Between Complex Systems and Software


Feature flags, also known as feature toggles, are a powerful technique that allows us to alter the behavior of our software systems without changing code.

Originating from the need for safer deployment practices, feature flags enable us to roll out features gradually, perform A/B testing (The desire to test different features and user experiences without deploying multiple versions of an application), and quickly revert changes if something goes wrong, all without deploying new code.

### A brief history
As the software industry embraced Agile methodologies and continuous integration/continuous deployment (CI/CD) practices, the need for mechanisms that could support these faster, iterative release cycles became apparent.

Feature flags emerged as a solution that could allow developers to merge their code into the main branch without immediately affecting the user experience, thus facilitating continuous deployment and testing in production environments.

Feature flags offer a way to mitigate deployment risks by allowing new features to be rolled out gradually to select user groups or environments, enabling teams to monitor performance and user feedback before a full release. This gradual rollout approach, also known as canary releases or phased rollouts, helps identify and fix issues without impacting all users.

### A Brief Walkthrough of how This Works
The frontend/backend code of our application makes a request to a dedicated server—known as the feature flag server.

This dedicated server responds with whether a feature should be enabled or disabled for the user, based on predefined criteria or configurations i.e. environment, user is a beta tester etc.

The dedicated server allows features to be toggled on or off in real-time, giving developers and project managers unparalleled control over the application’s functionality and user experience.

### Benefits of Using Feature Flags
The strategic use of feature flags brings significant advantages:

1. They allow for safer deployments by enabling feature testing in live environments without impacting all users. This peace of mind is invaluable.

2. Feature flags facilitate A/B testing, helping us make data-driven decisions about UI/UX and other key features. Moreover, they make it easy to enable or disable features without a full deployment cycle, offering flexibility that's crucial in today's fast-paced development environments.

3. Feature flags play a crucial role in defensive testing, a pragmatic approach to anticipate and mitigate risks preemptively. By selectively enabling features for certain user segments or environments, developers can vigorously test the stability and performance impacts of new functionalities under controlled conditions.

### Potential Challenges
Feature flags, while instrumental in enhancing flexibility and control in software deployment and testing, can lead to complications if not managed with discipline. When the use of feature flags becomes widespread within a software project without a cohesive management strategy, they can inadvertently contribute to a form of technical debt. Here's how this occurs:

1. Over time, as more feature flags are introduced into the system, the complexity of managing which features are on or off grows exponentially. This complexity not only makes it harder to understand the current state of the application but also increases the risk of errors or unexpected behavior when multiple flags interact in unforeseen ways.

2. Each feature flag, while useful during its active development and roll-out phases, requires ongoing maintenance. This includes updating the flag's parameters, ensuring compatibility with new features, and monitoring its performance implications. Without diligent management, the cost of maintaining these flags can escalate, consuming resources that could be better used elsewhere.

3. As features are progressively rolled out, tested, or rolled back, feature flags that are no longer needed should ideally be removed from the codebase. Failure to do so results in obsolete flags cluttering the code, making it more challenging to read and understand. This clutter, or code bloat, not only makes future development slower but can also introduce bugs due to confusion over which flags are active and which are remnants of past experiments.

4. In certain scenarios, feature flags can also impact the performance of the application. Checking the status of multiple feature flags, especially if this involves network calls or database lookups, can introduce latency. Furthermore, conditional logic based on feature flags can make some sections of the code less efficient or harder to optimize.

### Mitigating these Issues
To mitigate these issues, adopting a disciplined approach to feature flag management is essential. These involve:

1. Setting clear guidelines for the lifecycle of a feature flag

2. Regularly auditing and cleaning up old flags

3. Using feature flag management tools that help track and control flags' states across different environments.
By addressing the potential for technical debt proactively, teams can continue to reap the benefits of feature flags without succumbing to their drawbacks.

### A Practical Demonstration
For this demonstration, we'll be using an open source tool called Flagsmith. You can find its repo [here.](https://github.com/Flagsmith/flagsmith)

We'll also be using nextjs as our frontend repo to which feature flags will be enabled and disabled. The flow is direct and may be reciprocated to other frameworks and languages.

First off, we'll deploy Flagsmith's dedicated server using docker. Therefore make sure you have docker installed.

1. Download/copy the [docker-compose file](https://github.com/Flagsmith/flagsmith/blob/main/docker-compose.yml) and save it to a location of your choice.
2. `cd` to that location and run `docker-compose up --build`. This command will build the docker images described in the dockerfile and run the Flagsmith server. Also, check the logs which specify your username and password to log into the flagsmith server.
3. Go to your nextjs repo and run `npm i flagsmith`. This command will install the flagsmith SDK for nodejs/nextjs.
4. In your layout.js/.tsx or App.js/tsx file add the Flagsmith provider:

```typescript jsx
<FlagsmithProvider
  options={{
  api:"http://127.0.0.1:8000/api/v1/",
    environmentID: "kdLeKodsffrtegaQuptXUJZwS", // Generated after running the flagsmith server. Check the logs after running the docker-compose command.
}}
flagsmith={flagsmith}
  >
  {children}
  </FlagsmithProvider>
```

5. For every component you now want to react to Flagsmith changes, use the hook:

```typescript jsx
const flagsmith = useFlagsmith();
```

6. To check if a feature is enabled/disabled:

```typescript jsx
flagsmith.hasFeature('feature_name');
```

You can find more documentation and examples in their [official documentation page.](https://docs.flagsmith.com/)

### Conclusion
In conclusion, feature flags represent a strategic tool in modern software development, offering the flexibility to test new features, perform A/B testing, and adopt a more dynamic deployment strategy. By enabling developers to toggle features in real-time without deploying new code, feature flags can significantly enhance the safety, speed, and efficiency of software delivery. However, as with any powerful tool, discipline in management is crucial to avoid potential pitfalls such as increased complexity and technical debt.

The practical demonstration with Flagsmith and Next.js serves as a tangible example of how easily feature flags can be integrated into your development workflow, making it clearer how this approach can be adopted across various technologies and frameworks. As we continue to push the boundaries of what's possible in software development, feature flags stand out as a critical enabler of agility and precision in delivering user-centric features and experiences.


Escaping Deployment Hell: How Feature Flags Can Transform Your Workflow


JavaScript, a language of many faces, often presents concepts that are deceptively simple yet profound in their implications. Among these, understanding equality checks and the nuanced landscape of JavaScript types stands paramount. Whether you're just dipping your toes into the vast ocean of JavaScript or you're a seasoned sailor navigating through the stormy quirks of the language, grasping these concepts is crucial. 

This blog post sets out to demystify a concept that's often sidelined but deserves a spotlight: JavaScript's identity and equality checks. Despite their critical importance, details on how these checks align with ECMAScript standards are not thoroughly addressed in either React's documentation or the MDN web docs. The ECMAScript Language Specification doc however, does a great job in explaining these concepts. You can find it [here](https://tc39.es/ecma262/multipage/#sec-intro) and hopefully build the next version of Chrome. 

Starting with foundational principles and progressively delving into complex nuances, we aim to explore this topic thoroughly, concluding with a hands-on example to reinforce your grasp on the subject.

### JavaScript Types: The Foundations
JavaScript, at its core, categorizes values into several types — `boolean`, `undefined`, `null`, `number`, `string`, `object`, and `symbol`. Each type serves a unique purpose:
<br/>

##### Boolean: 
Represents a logical entity, having two values: true or false.
<br/>

##### Undefined: 
Denotes a variable that has been declared but not yet assigned a value.
<br/>

##### Null: 
Represents the intentional absence of any object value. It's often used to indicate that a variable should hold an object but currently does not. 
<br/>

##### Number: 
Holds numeric values. JavaScript uses floating-point arithmetic for its numbers.
<br/>

##### String: 
Represents textual data. It's a sequence of characters. Practically every primitive has string representation of itself;.
<br/>

##### Object: 
Unlike the other types that are primitive, objects are collections of properties.
<br/>

##### Symbol: 
Introduced in ES6, symbols are unique and immutable data types used as identifiers for object properties.
<br/>

##### Hear ye, hear ye!
##### Understanding these types is the foundation upon which we build the concept of equality in JavaScript.
<br/>

### The Enigma of Null and Undefined

Both `null` and `undefined` denote the absence of value in a variable, but they are used in subtly different contexts. `undefined` is the JavaScript engine's way of saying, "This variable not only has no value, but also might not even exist (no name has been reserved in memory for that variable)." In contrast, `null` is a deliberate assignment by the developer, saying, "This variable presently has no value but specifically holds an empty or non-existent object. So `null` is an actual value, like `number` or `string`, the only difference being that the variable has no value by design (a name will be reserved in memory for that variable). "
<br/>

### Equality Checks: "== vs ==="
JavaScript offers two operators for equality checks: `==` (abstract equality) and `===` (strict equality).

- `==` checks for equality after performing any necessary type conversions. It means if the values being compared are of different types, JavaScript will try to convert them to a common type before making the comparison. So `"1" == 1` will return true since the value 1 which is a number (no double quotes), will be converted to the value 1 as a string (with double quotes). 
- `===`, or strict equality, checks for value equality without any type conversion. If the values being compared are of different types, the equality check immediately returns false. So in the previous example `"1" === 1` will return false since `string` and `number` are different types.

Understanding these distinctions is vital as they often lead to unexpected results, especially for beginners.
<br/>

### Identity vs. Equality: The Subtlety
In JavaScript, values can either have identity (complex data types like objects, where the value is a reference to a location in memory) or be without identity (primitive values). PAUSE.

What in Sam's hill does it mean to have a value with or without identity? Well, let's dive into that mystery together.

A value with identity basically means a value that includes an extra property whose value is unique to that specific value. Same way your GitHub username is unique to you, and there can never be a situation where someone else might happen to have another account using your username.

A value without identity is a value doesn't have this extra property that declares its uniqueness.

So to put this into even better context, when you declare a variable whose value is an object the object automatically becomes a value with identity, the unique property being the memory location of the variable:
```javascript
const variable = {name: "I am an object"};
```

When you however declare a variable whose value is a primitive, the value itself is the unique property (instead of the memory location):
```javascript
const variable = 1;
```

When comparing primitive values (`string`, `number`, `boolean`, `undefined`, `null`, `symbol`), JavaScript checks their equality by value (which now in this case, is the value itself). However, when comparing objects, JavaScript checks whether they reference the same location in memory — which now is the property that uniquely identifies the object.

So when checking for equality for objects, comparing two objects with the exact same properties and the exact same values for those properties will still return false since these objects have different locations in memory.

```javascript
const myObjectOne = {name: "I am unique!"};
const myObjectTwo = {name: "I am unique!"}
console.log(myObjectOne === myObjectTwo); // returns 'false' since each variable has it's own memory location.
console.log("-----------Boundary----------");

const myObjectOneAgain = {name: "I am unique!"};
const myObjectTwoAgain = myObjectOneAgain;
console.log(myObjectOneAgain === myObjectTwoAgain);// returns 'true' since the second variable's value is the first variable's memory location.
// So basically, the value of [myObjectTwoAgain] resolves to the memory address of [myObjectOneAgain].
console.log("-----------Boundary----------");; // prints out 'false' since each variable has it's own memory location.
```
<br/>

### Practical Application: useObjectEqualityChecker
Now, let's blend theory with practice through a React hook, `useObjectEqualityChecker`, designed to check object equality by comparing their key-value pairs rather than their memory reference.

```typescript
import { useMemo, useRef } from "react";

export type ObjType = { [key: string | number | symbol]: any; };

const useObjectEqualityChecker = (obj: ObjType) => {
  const refObject = useRef<ObjType>({});

  return useMemo(() => {
    if (obj) {
      const keys = Object.keys(obj);
      let valuesMatch = true;
      keys.forEach((k) => {
        if (refObject.current[k] !== obj[k]) {
          valuesMatch = false;
        }
      });
      if (!valuesMatch) {
        refObject.current = { ...refObject.current, ...obj, };
      }
    }
    return refObject.current;
  }, [obj]);
};

export default useObjectEqualityChecker;
```

This hook demonstrates a way to check equality beyond mere reference checking. It iteratively compares each key-value pair of the current and previous objects, updating the reference only when a discrepancy is found. This approaches equality checking with more nuance, especially valuable in React, where re-renders should be minimized for performance.

If you want even more sophisticated equality checks like deep comparisons, there's the: [lodash.iseQual](https://lodash.com/) library that'll do this for you out of the box. At least now you do have a bit of clarity on what's happening under the hood, so there's no harm in using a library.



### Wrapping Up

Equality in JavaScript is a multifaceted concept, laced with subtleties that can lead to nuanced bugs and performance issues if misunderstood. By unraveling these intricacies, from different types to identity checks, and applying them in practical scenarios like the `useObjectEqualityChecker` hook, developers can write more efficient and bug-free code. Understanding equality deeply not only aids in mastering JavaScript but also in harnessing the full power of frameworks such as React.

And there you have it — a journey from the foundational to the advanced, all within the realm of JavaScript's equality checks. May this guide serve as a beacon, illuminating the path for both neophytes and veterans of JavaScript alike. 

Cheers!


JavaScript Equality Under the Lens: Enhancing React's Dependency Checks with an Innovative Hook Example

JavaScript Equality Under the Lens: Enhancing React's Dependency Checks

In today's fast-paced digital world, efficient data handling is crucial. Whether you're processing massive amounts of data, streaming real-time updates, or delivering content to multiple clients concurrently, traditional methods can quickly become overwhelming. That's where the Web Streams API comes in. It offers a solution that allows you to efficiently stream data without overloading your server's memory. The term 'Streams' means exactly what it sounds like: A stream of flowing data. We then obtain this data in bits as it continuously streams towards us. Let's explore this powerful API through the lens of a real-world example: delivering over 6000 log lines concurrently across 20 tabs.

### A Brief History Lesson
Imagine you have a large log file with over 6000 lines of valuable information. Traditionally, fetching this data and displaying it on a webpage would require loading the entire file into memory, which could slow down your server and impact performance. However, with the Web Streams API, you can stream the data on-demand, only fetching what you need when you need it. This not only saves memory and server (and client) resources, but also ensures a seamless user experience, as the logs can be displayed in real-time without delay.

The applications of the Web Streams API extend far beyond log files. Consider scenarios such as real-time analytics dashboards, chat applications, or even media streaming services. By using the Web Streams API, you can efficiently handle large volumes of data and provide responsive and dynamic experiences to your users.

### An Example Using a Large Log File (6000+ lines)
To demonstrate the power of the Web Streams API, let's dive into some code snippets. We'll be using Next.js and the app router for this example, but the Web Streams API can be used in various frameworks and environments (it's a standardized web API. You can read more about it [here](https://developer.mozilla.org/en-US/docs/Web/API/Streams_API)).

TLDR;
To see the code in action and explore a comprehensive example project, check out the GitHub repository: 
[streaming-text-from-file-example](https://github.com/TylerMutai/streaming-text-from-file-example).

The example repository has only 3 important files: app/api/route.ts, app/logs/page.jsx and app/page.jsx.
Let's go over the code of these three in more detail.

### app/api/route.ts
```typescript
import { NextResponse } from "next/server";
import fs from "node:fs";
import path from "node:path";

/**
 *
 * @param duration - milliseconds to wait before resolving the promise.
 */
function sleep(duration = 2000) {
  return new Promise((resolve) => {
    setTimeout(resolve, duration);
  });
}

async function readFile(filePath: string, start: number, end: number) {
  const readStream = fs.createReadStream(filePath, {
    start,
    end,
  });

  return new Promise<Buffer>((resolve) => {
    readStream.on("data", (data) => {
      resolve(data as any);
    });
  });
}

function getFileReaderConfig() {
  // File being read. the file contains at least 6 thousand lines of logs.
  // They'll all be efficiently streamed to the frontend without overusing your server's memory.
  // Open multiple tabs using the same URL and see for yourself.
  const filePath = path.join(process.cwd(), `logs/2024-02-13.log`);
  const fileStats = fs.statSync(filePath);
  const byteLengthToRead = 16000; // represents 16KB. There's no reason for this, I just chose the number since it
  // showcases streaming better.

  const start = 0;
  let end = start + byteLengthToRead;

  if (end > fileStats.size) {
    end = fileStats.size;
  }

  return { filePath, start, end, byteLengthToRead, fileSize: fileStats.size };
}

async function* makeIterator(
  filePath: string,
  start: number,
  end: number,
  byteLengthToRead: number,
  fileSize: number,
): any {
  if (start >= end) {
    // we have read the whole file. save last read bytes and return.
    return;
  }
  // This is just added here to better showcase streaming on the browser. You'll be able to see data being populated.
  // Feel free to remove this line to use streaming at its best.
  await sleep(10);

  // The reason there's a start + 1 here is because we end up re-reading an extra byte everytime, duplicating
  // the last character read twice.
  yield await readFile(filePath, start + 1, end);

  // move [byteLengthToRead] bytes behind.
  const _start = end;
  let _end = end + byteLengthToRead;

  // if moving [byteLengthToRead] bytes behind becomes less than [ceiling], then move our
  // [_start] to [ceiling].
  if (_end > fileSize) {
    _end = fileSize;
  }
  yield* makeIterator(filePath, _start, _end, byteLengthToRead, fileSize);
}

function iteratorToStream(iterator: any) {
  return new ReadableStream({
    async pull(controller) {
      const { value, done } = await iterator.next();

      if (done) {
        controller.close();
      } else {
        controller.enqueue(value);
      }
    },
  });
}

export async function GET() {
  try {
    const { filePath, start, end, byteLengthToRead, fileSize } =
      getFileReaderConfig();

    const iterator = makeIterator(
      filePath,
      start,
      end,
      byteLengthToRead,
      fileSize,
    );
    const stream = iteratorToStream(iterator);

    return new Response(stream, {
      headers: {
        "Content-Type": "text/html; charset=utf-8",
      },
    });
  } catch (e) {
    return NextResponse.json(
      {
        message: `A server error occurred. Contact support. ${(e as any)?.message}`,
        code: 500,
      },
      {
        status: 500,
      },
    );
  }
}
```
This file represents a route handler (the new jargon used by App Router in place of api routes that were used in the Pages Router. Read more here: [Next Route Handlers](https://nextjs.org/docs/app/building-your-application/routing/route-handlers))

We have a 'GET' method that handles 'GET' requests to that route. We have a helper funtion `getFileReaderConfig()` that reads information about the file we're about to fetch (rather than reading the file itself). Here, we get important information like the file's size which we'll use to calculate the chunks we'll be serving back to the client.

We then have the `makeIterator` function, which is a generator function (as declared by `function*`). A generator function has the ability to persist its context across calls. This means the function can exit, and when called next, remembers the 'data' it had, and resume from it. You can [read more here](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/function*). 

This is important since we need a way to recollect the last chunk we served to the client, and to which boundary that chunk got us. I.e. if we're serving a file of say 1000 bytes, and we have defined our chunk size as 100, and we have served the chunk 5 times, we should serve only the remaining chunk which is 500 bytes (100 * 5 chunks = 500 bytes. 1000- 500 bytes = 500 bytes).

The `makeIterator` function declares the `fileSize` (maximum bytes to read), the `start` (the start boundary, which initially is '0') and the `end` positions (which is initially the `byteLengthToRead` which is basically a 'chunk' of specific length, in this case, 16000 bytes). We then use the sliding window technique, where we continuously move the 'chunk' towards the end of the file until we have read the whole file.

We then read the file itself using `fs.createReadSteam` which has an extra argument called `options` which allow you to specify the start and end positions of the file you intend to read.

### app/logs/page.tsx
```typescript
"use client";
import { useCallback, useEffect, useMemo, useRef, useState } from "react";

const anchorForScrollId = "anchor-to-tell-document-to-scroll-to";
export default function Home() {
  const [streamingLogs, setStreamingLogs] = useState<string>("");
  const [errorMessage, setErrorMessage] = useState<string>("");
  const timeoutHandle = useRef<NodeJS.Timeout>();
  const isFetchOngoing = useRef(false);

  const getLogs = useCallback(async () => {
    if (isFetchOngoing.current) {
      return;
    }
    isFetchOngoing.current = true;
    setStreamingLogs("");
    try {
      const response = await fetch(`/api/logging`);
      if (response.ok) {
        const textDecoder = new TextDecoder();
        const reader = response?.body?.getReader();

        if (reader) {
          while (true) {
            const { value, done } = await reader.read();
            if (value) {
              const stringData = textDecoder.decode(value);
              setStreamingLogs((l) => `${l}${stringData}`);
            }
            // if done, call getLogs again at some point in future for more data.
            if (done) {
              // reset the logs to an empty string so that for the next fetch, we don't duplicate strings.
              clearTimeout(timeoutHandle.current);
              timeoutHandle.current = setTimeout(getLogs, 100000);
              isFetchOngoing.current = false;
              return;
            }
          }
        } else {
          const json = await response.json();
          setErrorMessage(
            `An error occurred:${json?.message}. You'll need to refresh this page.`,
          );
        }
      }
    } catch (e) {
      setErrorMessage(
        `An error occurred: ${(e as any)?.message} ${(e as any)?.response?.message}. You'll need to refresh this page.`,
      );
    }
  }, []);

  useEffect(() => {
    getLogs().then();

    const _handle = timeoutHandle.current;
    return () => {
      clearTimeout(_handle);
    };
  }, [getLogs]);

  useEffect(() => {
    document.getElementById(anchorForScrollId)?.scrollIntoView({
      behavior: "smooth",
    });
  }, [streamingLogs]);

  const logs = useMemo(() => {
    return streamingLogs?.split("\n");
  }, [streamingLogs]);

  return (
    <div className="flex flex-col w-full h-[100vh] p-10 overflow-auto">
      <h5 className="w-full pb-4 border-b border-secondary-color-100 border-solid">
        Logs should automatically start appearing below.
      </h5>
      <h5 className="text-red-700">{errorMessage}</h5>
      {logs?.map((l, idx) => (
        <h6 key={idx} className="w-full">
          {l || <br />}
        </h6>
      ))}
      <div id={anchorForScrollId} />
    </div>
  );
}
```
This file is a React component that does the heavy lifting on the client side. In this file, we make a network request to our logging api that we just defined in the previous code snippet. We then obtain a 'ReadableStreamDefaultReader' object by calling `response.body.getReader()` which will help us read single chunks as they're made available in the stream. We then convert these byte streams into strings using the `TextEncode API` and render this string on the UI.

Notable mentions are the `isFetchOngoing` ref and the `timeoutHandler`. The `isFetchOngoing` ref stores a boolean value to indicate whether there's already a stream currently being consumed. If it's the case, it prevents duplicate requests being sent that will then read independent streams and mess up the UI rendering. The `timeoutHandler` stores a reference to the `setTimeout` function we call (since we end up polling the server every 100 seconds) that allows us to clear the timeout event in cases where out component unmounts and setTimeout was already fired.

### app/page.tsx
```typescript
"use client";

import { useEffect } from "react";

export default function Home() {
  // open 20 tabs instantaneously to see whether the server will handle all streams.
  // You will need to 'allow all popups' in your browser to see this in effect.
  useEffect(() => {
    let tabs = 20;
    while (tabs) {
      window.open("/logs", "_blank");
      tabs--;
    }
  }, []);

  return (
    <h2 className="p-10 m-10 w-full flex flex-row- items-center justify-center">
      Nothing&apos;s happening? Make sure to enable popups in your browser!
    </h2>
  );
}
```
This is the page that is now rendered on the '/' (or index) route. We open 20 tabs concurrently each having its own established stream (the previous code snippet that handles streaming of data from the backend) to see how the server behaves under this pressure. 

The server does just fine, streaming all 6000+ lines back to all 20 clients (as represented by the tabs).


### Conclusion
The Web Streams API opens up new possibilities for handling large volumes of data efficiently. By streaming data on-demand instead of loading it all into memory, you can provide responsive experiences to your users without sacrificing server performance. Whether you're dealing with log files, real-time analytics, or media streaming, incorporating the Web Streams API can greatly enhance your application's capabilities.

In this blog post, we've explored the basics of the Web Streams API, provided practical examples and code snippets, and highlighted its significance in today's data-driven world. Now it's time for you to dive in, experiment with streaming data, and unlock new possibilities for your web applications.

Remember, efficient data handling is the backbone of modern web applications. Embrace the power of streaming data, and unlock a whole new level of performance and user experience!


Explore the powerful Web Streams API with this guide. Learn to handle 6000+ log lines across 20 tabs concurrently, boosting web performance and user experience.

Web Streams API in Action: Delivering 6000+ Log Lines Concurrently Across 20 Tabs


Dare I say, we've all been there - staring at our perfectly tailor-made Figma designs, only to dread the impending ordeal of translating it pixel-for-pixel in code. The challenge often intensifies when grappling with SVGs, particularly when trying to tweak them to react to user interactions. But worry not - I have a solution worth considering.

### The SVG Ordeal in React

SVGs, with their crisp edges and responsiveness to scaling, have a special place in our developer hearts. The love story, however, has its occasional frictions when you wish your SVGs to be more "reactive"- like changing color on hover, growing, shrinking, coming alive. You end up scrambling for icon libraries, like react-icons, but good luck finding the perfect match for your designs there!

### The Case of UI Libraries

Strapped for alternatives, you resort to using UI libraries. Material UI? Been there. Chakra UI? Done that. Yet, despite bundling your projects with these bulky companions, the SVG remains seemingly aloof. React even comes with the ability to import SVGs directly as React Components. Be it a massive UI library or React's native functionality, they both skip a vital step - processing the SVGs to relinquish their hard-coded aesthetic attributes. The SVGs, regardless of their fancy new wrappers, stay indifferent to the dynamically changing environment.

The colors, strokes, fills and other hardcoded properties remain the same; the SVG refuses to react. The upshot? Your project has accumulated unnecessary weight, and the SVG hasn't budged from its original state.

### Breaking Free: My Alternative Approach

From my trenches, I offer a counterproposal: a streamlined, lightweight alternative that abolishes dependencies and ignores pre-processed icons. My solution transforms your beloved SVG images into stateful, responsive React components, honoring their parent properties like color, font size, etc., and bringing your designs to life with precision. It's also not a library, not a dependency, nor a package - it's literally an approach, a step by step you can try out yourself.

And the best part? You can clone my GitHub repo, and do with the source code as you please. Feed it your SVGs, the tool works its magic (which really isn't anything sophisticated), and out comes a neat zip file with your tailor-made, ready-to-use React SVG components.

### The Walkthrough: A Cake Walk

Here's the [link to the GitHub repo](https://github.com/TylerMutai/svg-to-react-component) for those of you who'd rather jump directly into the code.

To illustrate the simplicity and effectiveness of this new approach, I present a step-by-step guide:

1. [Clone the GitHub repo](https://github.com/TylerMutai/svg-to-react-component) and follow the ReadMe instructions (which is basically `npm install` and `npm run build` and `npm run start`).
2. Upload your SVG images in the interface provided – the exact ones taken from your Figma designs - and click 'Convert'.
3. As the tool processes the SVGs, it automatically attends to all necessary preprocessing steps. It strips them of any hardcoded aesthetic values that usually resist changes, priming them for perfect integration into your project. Since this is just a POC, the tool is changing only the 'fill', 'stroke' and 'strokeWidth' properties, which I believe are enough for this scenario.
4. Download the resulting zip file containing your fully prepped React SVG components.

Embed them in a button, a div, an anchor, or whatever element their new home is. Write your CSS. Here's a sample:

```css
.parent {
  color: red;
}

.parent:hover {
  color: blue;
}
```

As long as your SVG is a child of 'parent', it'll change from red to blue.

### Proving the Pudding

Try it for a spin. Hover on a link – the SVG responds! Resize the parent – the SVG scales seamlessly. Change the text color – similar to its text siblings, it changes color correctly matching parent properties. In my opinion, this method significantly elevates your SVG implementation in React. By abolishing the complexity of integrating additional libraries and detaching your SVGs' visual attributes from their innate structure, it paves the way for a powerful advantage. Now, your SVGs can be styled directly with CSS, enabling them to respond and behave just like their component siblings within React's ecosystem.

(__A side note, I'm still in disbelief that this approach hasn't yet been attempted by anyone. I did my fair share of ransacking the internet, and didn't find any satisfactory solutions, including React's own built-in approach. I'm very open to having my mind changed.__)

### Feedback: A Milestone, Not the Destination

As excited as I am about this solution, I recognize it as an evolving proposition. Therefore, your perspective matters greatly. Questions, insights, alternatives, or a simple pat on the back would be invaluable. After all, progress in our developer community is driven by dialogues and shared genius.

Here's throwing a hat in the SVGs' ring in React. It's not a magic bullet or a universal panacea, but with your help, it can be a steady stride in aligning our SVGs to our tasks. I also believe the application of this approach isn't just restricted to React. This approach can be adapted to Flutter, Vue, and any other declarative framework.

Happy coding!


An alternative approach to reactive, responsive and customizable SVGs in React without needing to compromise nor install bulky libraries

Animating SVGs in React: Breathe Life into Your Images


Hello once again my fellow code-based creature enthusiasts!

Have you ever felt that managing pull requests can feel like navigating a complex, never-ending labyrinth where each
exit only leads to another room filled with code rather than a solution? Well, if you are nodding along or sighing
dramatically, we are about to dive deep into a pool of AI-driven pull request management tools. Buckle up, as we pit our
open-source underdog, CodiumAI's PR-Agent, against the reigning heavyweight, GitHub Copilot.

Now, just to be honest and clear: PR reviews can be as nerve-wracking (or more) as defusing an oddly colored blue-wired
bomb - under the camera of a Nokia Simens phone (for those born later, a 1MP camera). But fear not, CodiumAI to the
rescue to ensure PR reviews instead feel like a breezy walk at a picturesque park, or a quick, efficient jog if you're
in Kenya and feel so inclined. In the world of GitHub, which usually is the mecca for the developers, PR-Agent swaggers
in like a strong contender.

Itchy to know more? Scroll away!

### Opening Act - What is Github Copilot and Codium's PR-Agent?

GitHub Copilot and Codium's PR-Agent are two powerful tools designed to assist developers and streamline their coding
process.

GitHub Copilot, powered by OpenAI, essentially acts as an AI pair programmer. It leverages the knowledge of billions of
publicly available code snippets to suggest completions and generate code based on the context of your work. So, while
working on a task, it may suggest potential implementations for functions, write new code, aid in implementing best
practices, and even speed up the process of writing repetitive or boilerplate code.

On the other hand, Codium's PR-Agent is a highly capable, open-source tool designed to tackle the challenge of reviewing
pull requests efficiently. It's equipped with an impressive set of commands making pull request management a smoother
ride. With features like automated PR description generation, review feedback, question answering, code suggestions,
changelog updates, similarity check with other issues, documentation addition, and custom label generation, it gives you
much more direct control over your PR reviews.

While GitHub Copilot lends a helpful hand during the coding phase, Codium's PR-Agent steps up the game during the often
overlooked, but crucial, code reviewing process. Imagine being able to instantly describe a pull request, give immediate
feedback, improve your codes, update your changelogs, finding similar issues, generate documentation, and even creating
custom labels all with simple commands. It offers a sophisticated suite of tools that enhance your code reviewing
capabilities, making it simpler, faster, and so much more efficient. No hidden subscription fees or any
obligations. What’s not to love?

So, while GitHub Copilot serves as your sidekick while coding, Codium's PR-Agent ensures your pull requests are of the
highest quality, making it an indispensable ally in your coding workflow.

### The Ins and Outs of CodiumAI's PR-Agent

It is famously said that heavy is the crown worn by the king. In the programming world, the crown often goes to
customization as the king. CodiumAI's PR-Agent understands this better than most. It lets you tweak settings with
developer-level precision, the kind you'd want in your personal relationships, but we will stay within safe limits on
that one.

Ever wanted a software helper who's palms you can grease to sometimes look the other way during your code analysis? Just
drop in an easy `ignore.toml`. PR-Agent looks the other way, no questions asked. Just pop `ignore.toml` in and make
PR-Agent ignore certain files.

But wait, there's more.

Let's take a detailed journey about how CodiumAI's PR-Agent fits in your project, by creating a hypothetical scenario
and walking through the process.

## An Illustrated Walkthrough With CodiumAI's PR-Agent

The scenario:

Imagine we have a simple React app that uses Chakra-UI, with a couple of pages -- Login.jsx and Register.jsx. We will
work with our local repo and install CodiumAI's pr-agent as per instructions
provided [here](https://github.com/Codium-ai/pr-agent/blob/main/INSTALL.md#run-from-source).

_(Disclaimer: No actual code was harmed in writing this post)_

Here’s a quick overview of the steps to follow (assuming that you've at least gotten as far as cloning the repo):

- Navigate to the repo folder that you've just cloned:

  ```bash
  cd pr-agent
  ```

- We create a Python virtual environment before running pip install:

  ```bash
  python3 -m venv venv
  ```

- Next, set up an account on [Pinecone](https://www.pinecone.io/), a vector database required by Codium. After logging
  in, copy your API key from the 'API Keys' section or create a new key.
- Then, edit `pr_agent/settings/.settings.toml`. If it doesn't exist, create it using the following commands:
  ```bash
  cp pr_agent/settings/.secrets_template.toml pr_agent/settings/.secrets.toml
  ```

Since you might end-up running into errors when trying to run the pr-agent locally (this is because the '.secrets.toml'
file contains all possible entries, including optional ones uncommented) just use my settings.toml which has been
cleaned up:

```bash
# QUICKSTART:
# Copy this file to .secrets.toml in the same folder.
# The minimum workable settings - set openai.key to your API key.
# Set github.deployment_type to "user" and github.user_token to your GitHub personal access token.
# This will allow you to run the CLI scripts in the scripts/ folder and the github_polling server.
#
# See README for details about GitHub App deployment.

[openai]
key = "<your openai api key here>"  # Acquire through https://platform.openai.com

[pinecone]
api_key = "f8fc9404-e934-43fd-8dcc-11f51d4387a8"
environment = "gcp-starter"

[github]
# ---- Set the following only for deployment type == "user"
user_token = "<your github personal access token here>"  # A GitHub personal access token with 'repo' scope.
deployment_type = "user" #set to user by default
```

That's everything to do with installing Codium's pr-agent by building from source. Now onto the repo with our
hypothetical scenario. [Clone this repo](https://github.com/TylerMutai/codium-pr-agent-react-example.git) to efficiently
follow along.
Then run the usual within the repo's folder:

```bash
npm i
```

- We then branch out for our new feature

```bash
git checkout -b feature/hebrew-lang-support-with-rtl
```

At this point, we assume the feature is already built.
Here's [the commit listing all changes.](https://github.com/TylerMutai/codium-pr-agent-react-example/commit/d65634acce251a82d33e6dea5168009c22942ecf)

And then push our code to our branch.

```bash
git add .
git commit -m "added hebrew language support and rtl support"
git push -u origin feature/hebrew-lang-support-with-rtl
```

Next, create a PR using the provided link in the response message.

With that done, let’s say we are introducing a new feature for Hebrew language support with RTL. With PR-Agent in our
toolkit, we are ready to tackle it head-on.

First things first, we want to test our CodiumAI's PR-Agent. To do that, navigate back to the directory you cloned your
pr-agent in (which was covered earlier in this blog):

```bash
cd pr-agent
```

We then activate python's virtual environment:

```bash
source venv/bin/activate
```

We use the `reflect` command that generates some interesting queries:

```bash
python -m pr_agent.cli --pr_url=https://github.com/TylerMutai/codium-pr-agent-react-example/pull/1  reflect
```

Voila! You get a response, automatically pushed to the PR as comments under your user. PR-Agents infers significant
changes to the code and highlights them, saving us from a deep dive directly into the 'files changed' tab (_cough_
senior developers _cough_).

Here's a snippet of the response (view it in
full [here](https://github.com/TylerMutai/codium-pr-agent-react-example/pull/1#issuecomment-1838951359)):

![reflect-codium-pr-agent.png](/images/posts/reflect-codium-pr-agent.png)

---

Feeling adventurous, we let AI take the wheel a bit more and asked it to show us possible improvements or changes that
can be made to our PR using the `ask` command:

```bash
python3 -m pr_agent.cli --pr_url=https://github.com/TylerMutai/codium-pr-agent-react-example/pull/1 ask "Show me what improvements/changes can be made in this PR"
```

This is where the fun really starts. The AI generates a list of detailed improvements that we might have overlooked
otherwise. Not just grammar and optics, but actual, substantial methodology and logic improvements. I'm smiling like a
proud parent here!

Here's a snippet of the response which you can
find [here](https://github.com/TylerMutai/codium-pr-agent-react-example/pull/1#issuecomment-1838971826):

![questions-codium-pr-agent.png](/images/posts/questions-codium-pr-agent.png)

---

Next, we let Codium do what it does best using the `/describe` command.

```bash
python3 -m pr_agent.cli --pr_url=https://github.com/TylerMutai/codium-pr-agent-react-example/pull/1 describe
```

This command blesses our PR with an introduction that pretty adeptly summarizes it. Applause, please!
[Here's the full description generated.](https://github.com/TylerMutai/codium-pr-agent-react-example/pull/1#issue-2024089862)

![describe-codium-pr-agent.png](/images/posts/description-codium-pr-agent.png)

---

Now comes the royal rumble, my favourite. We put CodiumAI's PR-Agent to the ultimate test – the `/review`. This not only
gives you a summary of the PR; it also analyses the level of effort required to review it, checks for security concerns,
and provides constructive feedback and code-specific suggestions. At this point I might as well review my own code!
Here's a sample:

![review-codium-pr-agent.png](/images/posts/review-codium-pr-agent.png)

---

Finally, we utilize the `/add_docs` command. This results in potential documentation being popped in directly within
your code, making it so much easier for your fellow team members. With this, you've practically got an entire PR review,
neatly wrapped and delivered to your developer doorstep with a bow on top!

![Proposed docs](/images/posts/proposed-docs-codium-pr-agent.png)
![Proposed docs two](/images/posts/proposed-docs-two-codium-pr-agent.png)

## In Conclusion...

Isn’t it amazing when machines do the tedious work for you, and you can just pop in with your insightful, intelligent
comments to seal the deal? Nothing short of what developers have always wanted. Optimize for 4 years, so that you save
an additional 3 hours every day. Well, with Codium's pr-agent, you only need 1 hour.

I mean, why slog away doing the grunt work when CodiumAI's PR-Agent can handle:

- Automating the creation of PRs with commands
- Facilitating preparation of structured and detailed descriptions
- Easy-peasy analysis of PRs
- Detailed suggestions for code improvements

And it does all of that while catering to each developer’s unique needs and control freak tendencies! It’s like having a
highly efficient, attentive, machine-based colleague who never goes on leave even on Fridays!

So, all you master tinkerers, all you fellow Dev Dudes and Dudettes, if you fancy diving head-first into the ocean of
customizable options, CodiumAI's PR-Agent is ever ready to bring out your programmer alter-ego.

Facing off against GitHub Copilot, PR-Agent shines through with its open-source label, ready to roll up its sleeves and
take your developer life to the next level. Contesting against GitHub Copilot, backed by conglomerate giant Microsoft,
PR-Agent with the strength of versatility is a reminder that the right tool is not always the biggest name but the one
that fits snugly into our unique developer lives.

Well, that's it for today folks. Until next time, keep your curly braces matched and your coffee strong!

Over and out.

Brian Baliach

---

PS: I genuinely hope this cheeky jaunt through the wonders of CodiumAI's PR-Agent and GitHub Copilot has left you with a
smirk and a better understanding of AI-driven pull request management tools. Just remember, the right choice is not
always the crowd-favourite, but the one that is custom-tailored for your unique programmer needs!

Catch more insights and all things code at my blog: https://balysnotes.com. Or don't, and miss out on some top-tier
tech wit. Your call!

Tally-ho, Brian


Navigating the Pull Request Maze with CodiumAI's PR-Agent: A Comparison to GitHub's Copilot


Imagine this: You're typing a friendly message in a WhatsApp group directed to one of your peers, and your fingers,
seemingly possessed by a digital imp, instead of typing 'far too nice', types 'fart oo nice'. Now everyone thinks you're
a creep. Frustrating, right? Fear not! Trie is the unseen sidekick behind your keyboard here to maintain your public
image.

### How 'Trie' Solves the Typo Trouble

Think of Trie as your personal autocorrect wizard. When you type a word, Trie steps in to predict what you're trying to
say. It's like having a smart assistant that finishes your sentences, but for typing!

Imagine you're typing "eleph." Trie jumps in and goes, "Ah, you must mean 'elephant'!" It's like magic—no more wrestling
with the keyboard to spell out every word perfectly.

### So, How Does It Work?

I'll do something different here. Instead of starting off with a written explanation, I'll do a visual walkthrough on
how the Trie works (yes, I also happen to be an artist). If the visual explanation doesn't work for you, feel free to
read on!

![trie diagram](/images/posts/trie_explanation.drawio.png "Trie Diagram")

Picture Trie as a giant family tree of words. Each letter in a word is like a family member. The journey from the root (
the grandparent) to the leaves (the kids) spells out a complete word.

Let's say you've typed "eleph." Trie checks its family tree and quickly realizes that "eleph" fits perfectly into the "
elephant" branch. Voila! It suggests the complete word without missing a beat.

### A Friendly Go-Away Message

This silent sidekick is also evident in other aspects of your life other than Autocorrect. An example most of us might
relate to: Browser History. So for those clandestine folks who toggle 'Incognito Mode' at those wee hours to remove
traces
of their online presence, think of the 'Trie' and it's selfless contributions next time you do this.

Feel free to share your feedback especially if something wasn't clearly explained!

If you are intrigued by the inner workings of Trie, and ready for a deep dive into the digital ocean (don't sue me,
digitalocean), keep reading. The next section is where we unveil the code sorcery behind Trie for our tech-savvy
friends. Also, if you have already read the previous blog titled: 'Unveiling the Trie: A Masterstroke for Efficient
String Searches', you can stop here (but you don't have to :-)).

Gentlemen, and women, and everyone else, shall we?

A brief introduction to Trie: it's a data structure (similar to an array, a list, a tuple etc.) that stores data as
a tree where the branch of the tree traces out the path to where the data is, i.e. the leaf nodes/child nodes.

Here's the link to
the [GitHub repository with the full implementation.](https://github.com/TylerMutai/coding-challenges-solutions/tree/main/js-coding-challenges-solutions/trie_dictionary)

Now, let's get hands-on with some JavaScript code!

Here's our Trie implementation:

```javascript
// Trie Node
function TrieEntry(word) {
  this.word = word;
  const words = [];
  for (let i = 0; i < 26; i++) {
    words.push(null);
  }
  this.words = words;
}

// Main Trie Dictionary
function TrieDictionary() {
  const letterToIndexMapping = {
    "a": 1,
    "b": 2,
    "c": 3,
    "d": 4,
    "e": 5,
    "f": 6,
    "g": 7,
    "h": 8,
    "i": 9,
    "j": 10,
    "k": 11,
    "l": 12,
    "m": 13,
    "n": 14,
    "o": 15,
    "p": 16,
    "q": 17,
    "r": 18,
    "s": 19,
    "t": 20,
    "u": 21,
    "v": 22,
    "w": 23,
    "x": 24,
    "y": 25,
    "z": 26,
  }

  this.root = new TrieEntry();

  this.insertWord = (word) => {
    const wordChars = word.toLowerCase().split("");
    let currWord = "";

    let currSlot = this.root;
    for (const char of wordChars) {
      currWord += char;
      // traverse our root adding each char.
      const idx = letterToIndexMapping[char] - 1;
      const arraySlotForChar = currSlot.words[idx];
      if (arraySlotForChar) {
        currSlot = arraySlotForChar;
      } else {
        currSlot.words[idx] = new TrieEntry(currWord);
        currSlot = currSlot.words[idx];
      }
    }
  }

  this.searchWord = (word) => {
    const wordChars = word.toLowerCase().split("");

    let currSlot = this.root;
    for (const char of wordChars) {
      const idx = letterToIndexMapping[char] - 1;
      const arraySlotForChar = currSlot.words[idx];
      if (!arraySlotForChar) {
        break;
      }
      currSlot = arraySlotForChar;
    }
    return currSlot.word === word;
  }

  this.printDictionary = () => {
    const printRecursively = (entry) => {
      if (!entry) return "";

      for (const en of entry.words) {
        if (en) {
          console.log("en: ", en);
          console.log("--------------------");
        }
        printRecursively(en);
      }
    }

    printRecursively(this.root);
  }
}
```

### Why Trie? A Performance Benchmark Comparison Between Trie and Linear Search

Feel free to run the code directly on your machine if you'd prefer a more hands-on approach.

#### 1. Dictionary Population (1049938 words)

Adding words to our Trie is a breeze. Here's the javascript code sample:

```javascript
const trieDict = new TrieDictionary();
const wordsDictionary = ["apple", "banana", "orange", /* ...a million more words */];

const start = new Date();
for (const word of wordsDictionary) {
  trieDict.insertWord(word);
}
const end = new Date();

const elapsedTime = (end - start) / 1000;
console.log(`Trie Population Time: ${elapsedTime} seconds`);
```

```bash
Dictionary population (1049938 words) timestamp start:  1699963909921
Timestamp after population:  1699963911482
Trie Population Time:  1.561
------------------------------------------------------------
```

In a mere 1.561 seconds, our Trie is ready to rock and roll with over a million words. Impressive, isn't it?

#### 2. The Great Search Showdown

Now, let's pit Trie against the traditional linear search in a battle for supremacy. Our contenders: Trie vs. Linear
Search!

```javascript
const wordsToSearch = ["apple", "programming", "debugging", /* ...4,829 more words */];

// Linear Search
const linearSearchStart = new Date();
for (const word of wordsToSearch) {
  // ... linear search logic
}
const linearSearchEnd = new Date();

// Trie Search
const trieSearchStart = new Date();
for (const word of wordsToSearch) {
  trieDict.searchWord(word);
}
const trieSearchEnd = new Date();

// Calculating and Displaying Results
const linearSearchTime = (linearSearchEnd - linearSearchStart) / 1000;
const trieSearchTime = (trieSearchEnd - trieSearchStart) / 1000;

console.log(`Linear Search Time: ${linearSearchTime} seconds`);
console.log(`Trie Search Time: ${trieSearchTime} seconds`);
```

Drumroll, please! The results are staggering:

- **Linear Search Time:** 8.839 seconds
- **Trie Search Time:** 0.004 seconds

Yes, you read it right! Trie search takes a mere fraction of a second, showcasing a jaw-dropping 2200X performance
boost!

### Use-Cases: 'Trie's' Playgrounds

#### 1. Autocomplete During Search

Ever wondered how autocomplete in search bars works seamlessly? Trie is the wizard behind the curtain, swiftly
predicting your next word as best as it can.

#### 2. Spell Checking

Tries are responsible for checking typos and spelling mishaps. It detects errors with the swiftness of a
well-optimized algorithm, making corrections before you finish typing "debug."

### Wrapping it Up: Trie and the Future of Performance

In the blink of an eye, Trie has transformed the landscape of our search experiences. With its unparalleled speed in
word insertion and instant searches, this data structure emerges as a strategic asset for software optimization.

As you navigate through the complexities of code, consider the Trie as a reliable companion, streamlining your path
through the intricate web of linguistic data.

Before we part ways, contemplate this: How might integrating Trie elevate the efficiency of your software projects?
Share your insights in the comments below! Stay tuned for our upcoming tech discussions, where we'll unravel more coding
intricacies.

Wishing you seamless coding journeys!


Autocomplete Demystified: A Common Use-Case of Trie


Welcome back, fellow developers! In our last rendezvous, we deciphered the intricacies of recursion, peeking into the call stack and heap dynamics. Today, we embark on a journey to unravel the mysteries of the Trie data structure, a silent hero in the realm of efficient string searches. So buckle up your code belts and let's dive in!

### The ABCs of Trie: What is it?

Imagine a dictionary, organized not alphabetically, but in a way that groups words by common prefixes. That's essentially what a Trie (pronounced "try") is—a tree-like data structure where each node represents a single character. It's a linguistic maze where the journey from the root to a leaf node spells out a complete word.

Here's the link to the [GitHub repository with the full implementation.](https://github.com/TylerMutai/coding-challenges-solutions/tree/main/js-coding-challenges-solutions/trie_dictionary)

Now, let's get hands-on with some JavaScript code!

Here's our Trie implementation:

```javascript
// Trie Node
function TrieEntry(word) {
  this.word = word;
  const words = [];
  for (let i = 0; i < 26; i++) {
    words.push(null);
  }
  this.words = words;
}

// Main Trie Dictionary
function TrieDictionary() {
  const letterToIndexMapping = {
    "a": 1,
    "b": 2,
    "c": 3,
    "d": 4,
    "e": 5,
    "f": 6,
    "g": 7,
    "h": 8,
    "i": 9,
    "j": 10,
    "k": 11,
    "l": 12,
    "m": 13,
    "n": 14,
    "o": 15,
    "p": 16,
    "q": 17,
    "r": 18,
    "s": 19,
    "t": 20,
    "u": 21,
    "v": 22,
    "w": 23,
    "x": 24,
    "y": 25,
    "z": 26,
  }

  this.root = new TrieEntry();

  this.insertWord = (word) => {
    const wordChars = word.toLowerCase().split("");
    let currWord = "";

    let currSlot = this.root;
    for (const char of wordChars) {
      currWord += char;
      // traverse our root adding each char.
      const idx = letterToIndexMapping[char] - 1;
      const arraySlotForChar = currSlot.words[idx];
      if (arraySlotForChar) {
        currSlot = arraySlotForChar;
      } else {
        currSlot.words[idx] = new TrieEntry(currWord);
        currSlot = currSlot.words[idx];
      }
    }
  }

  this.searchWord = (word) => {
    const wordChars = word.toLowerCase().split("");

    let currSlot = this.root;
    for (const char of wordChars) {
      const idx = letterToIndexMapping[char] - 1;
      const arraySlotForChar = currSlot.words[idx];
      if (!arraySlotForChar) {
        break;
      }
      currSlot = arraySlotForChar;
    }
    return currSlot.word === word;
  }

  this.printDictionary = () => {
    const printRecursively = (entry) => {
      if (!entry) return "";

      for (const en of entry.words) {
        if (en) {
          console.log("en: ", en);
          console.log("--------------------");
        }
        printRecursively(en);
      }
    }

    printRecursively(this.root);
  }
}
```

### Why Trie? A Performance Benchmark Comparison Between Trie and Linear Search

Feel free to run the code directly on your machine if you'd prefer a more hands-on approach.

#### 1. Dictionary Population (1049938 words)

Adding words to our Trie is a breeze. Here's the javascript code sample:

```javascript
const trieDict = new TrieDictionary();
const wordsDictionary = ["apple", "banana", "orange", /* ...a million more words */ ];

const start = new Date();
for (const word of wordsDictionary) {
  trieDict.insertWord(word);
}
const end = new Date();

const elapsedTime = (end - start) / 1000;
console.log(`Trie Population Time: ${elapsedTime} seconds`);
```

```bash
Dictionary population (1049938 words) timestamp start:  1699963909921
Timestamp after population:  1699963911482
Trie Population Time:  1.561
------------------------------------------------------------
```

In a mere 1.561 seconds, our Trie is ready to rock and roll with over a million words. Impressive, isn't it?

#### 2. The Great Search Showdown

Now, let's pit Trie against the traditional linear search in a battle for supremacy. Our contenders: Trie vs. Linear Search!

```javascript
const wordsToSearch = ["apple", "programming", "debugging", /* ...4,829 more words */ ];

// Linear Search
const linearSearchStart = new Date();
for (const word of wordsToSearch) {
  // ... linear search logic
}
const linearSearchEnd = new Date();

// Trie Search
const trieSearchStart = new Date();
for (const word of wordsToSearch) {
  trieDict.searchWord(word);
}
const trieSearchEnd = new Date();

// Calculating and Displaying Results
const linearSearchTime = (linearSearchEnd - linearSearchStart) / 1000;
const trieSearchTime = (trieSearchEnd - trieSearchStart) / 1000;

console.log(`Linear Search Time: ${linearSearchTime} seconds`);
console.log(`Trie Search Time: ${trieSearchTime} seconds`);
```

Drumroll, please! The results are staggering:

- **Linear Search Time:** 8.839 seconds
- **Trie Search Time:** 0.004 seconds

Yes, you read it right! Trie search takes a mere fraction of a second, showcasing a jaw-dropping 2200X performance boost!

### Use-Cases: Trie's Playgrounds

#### 1. Autocomplete Awesomeness

Ever wondered how autocomplete in search bars works seamlessly? Trie is the wizard behind the curtain, swiftly predicting your next word with unparalleled speed.

#### 2. Spell Checking Sorcery

Trie is your vigilant guardian against typos and spelling mishaps. It detects errors with the swiftness of a well-optimized algorithm, making corrections before you finish typing "debug."

### Wrapping it Up: Trie and the Future of Performance

In the blink of an eye, Trie has transformed the landscape of our search experiences. With its unparalleled speed in word insertion and instant searches, this data structure emerges as a strategic asset for software optimization.

As you navigate through the complexities of code, consider the Trie as a reliable companion, streamlining your path through the intricate web of linguistic data.

Before we part ways, contemplate this: How might integrating Trie elevate the efficiency of your software projects? Share your insights in the comments below! Stay tuned for our upcoming tech discussions, where we'll unravel more coding intricacies.

Wishing you seamless coding journeys!


Unveiling the Trie: A Masterstroke for Efficient String Searches

If you’ve been following me, you might recall a harrowing tale of a developer coming to grips with recursion. The aptly titled 'Decoding Recursion: The Concept That Had Me Stumped'. But your cogitation and feedback suggest we dig a tad deeper. Brave ones have emerged requesting a practical walkthrough of recursion. In this sequel, we'll take a hands-on approach to recursion using the call stack/heap.

Let's hoist the mainsail and dive in.

### Under the Hood: What Happens When Functions Run?

So, matcha latte in hand, (or is it an afternoon beer?), let’s unravel what’s going on 'under the hood' when functions are involved. Every function in its operation has an allocated memory space. Avoiding the technical jargon for the sake of our shared sanity, let's visualize this as tiny compartments in your computer's brain.

When a function runs, it's like the computer opens that compartment and dumps everything the function needs inside – the variables, calculations, and that little reminder of your lunch break at 12 pm.

This "compartment" goes by a couple of names in the programming world: Stack and Heap. While they are both for storage, they differ slightly in usage. The stack is used for static memory allocation and the heap for dynamic memory allocation.

### Enter Recursion: A Practical Look

Since we're all here for recursion, let's take a prime example - factorial computation. A factorial is the product of a number (n) and all the integers below it. Remember 5! (factorial) equaling 5x4x3x2x1? Yes, that one.

In code form, a basic recursive function for a factorial would look something like (javascript this time, cause turns out, it has a much larger audience):

```javascript
function factorial(n) {
  if (n === 1) {
    return 1;
  } else {
    return n * factorial(n - 1);
  }
}
```

Now, let's visualize this on the call stack.

Every recursive call will add a layer to the stack. It's like a pile of dishes. The first call (n=5) is the first plate. Then n=4, then 3, 2, and finally 1. After that, the result bubbles up, washing the dirty dishes one by one from the top until you reach the clean plate at the bottom, i.e., the final answer.
Remember that in heaps/stacks, data is added in a first come last serve basis.
Here's a visualization:

| Heap         | Result | Code explanation                                                                        |
|--------------|--------|-----------------------------------------------------------------------------------------|
| n==1         | 1      | base case: (n==1) return 1;                                                             |
| n==2         | 1 * 2  | n is 1 (n * factorial(n-1) which comes from previous call above. result then becomes 2. |
| n==3         | 2 * 3  | n is 2, which again comes from previous call above. the result becomes 6.               |
| n==4         | 6 * 4  | n is 6, which again comes from the call above. the result becomes 24.                   |
| n==5         | 24 * 5 | n is 24, the result becomes 120.                                                        |
| final result | 120    | This is what is finally returned. 120.                                                  |


### Stack Overflow: Recursion's Gotcha!

Stack overflow – it isn't just a popular website, folks; it's recursion's Achilles heel. This happens when our recursion burrows too deep, causing the application to run out of memory space. This is because every OS assigns a running program a fixed memory allocation (i.e. 4MB worth of space). If you exhaust this space, you then might get a stack overflow /segmentation fault error. There goes the stack of dirty dishes tumbling down!

This potential drawback highlights how deep understanding of recursion can help developers optimize for performance.

### Wrapping It Up

There you have it - a code soup stirred with the call stack/heap and topped off with a sprinkle of recursion. Hopefully, this has helped demystify the concept a bit. But who knows? Perhaps our odyssey is just starting - linked lists, binary trees, sorting algorithms - all served with that recursion spice, are waiting!

Remember, fellow devs, recursion is a powerful tool and understanding how it works at every level aids in writing better, cleaner, more intuitive and more efficient code... even if it stumps us occasionally!

As always, sail through your coding voyage, one recursion at a time.



Unraveling Recursion: A Practical Look at the Call Stack and Heap

Microservices might be the heroes of scalable and reliable applications today, but let's shift gears and talk about a concept in the software development landscape that is as mind-bending as it is powerful: recursion.

As someone who enjoys the adrenaline rush of conquering coding challenges, I've come across various concepts that are the bread and butter of computer science. Some felt like meeting an old friend on the first day of school—familiar and comforting. Others were like that one high school math problem that kept me up at night. Enter recursion.

Recursion, to put it simply, is like a mythic serpent that swallows its own tail—a function that munches on itself to get to a result. My initial thought was, "Why suffer through this mind-boggling feat when a good ol’ loop could suffice?" But then, as I chewed over recursion, the mist started to fade.

### Imagine This...
Let's play with recursion's call stack visualization. Suppose a group of kids are playing with a neat stack of blocks. Each block has a unique number. The rule is: a player can only place a block on the stack if it decreases the previous number by one. The game starts with 10 and ends with 1—a stack of decrescendoing integers.

So, what does this have to do with recursion? Each block placement is akin to a function call in recursion. You place one block (function call), then another (another call), and another, till you reach the edge case — in our scenario, the number 1 block. That's when the players start to cheer and pass the stacks backwards, unwinding them while they execute some extra celebration (in our case, processing returns).

### Dividing to Conquer
Remember those divide-and-conquer strategies in ancient warfare history? They work magic in recursion too. A problem must be conquerable by breaking it down into smaller, more manageable versions of itself.  Simply take down towers one by one until the main castle surrenders.

### A Quick Dive Into Code
Let's demonstrate recursion with a problem that even I, a seasoned loop-lover, found elegant in its recursive cloak — a classic problem: calculating the factorial of a number.

The factorial of a number n (denoted as n!) is the product of all positive integers up to n. For instance, the factorial of 4 is 4! which equals 4 * 3 * 2 * 1 = 24.

A looping method would have us running circles. Here's a python example (python because, from my perspective, is closest to English):

```python
def factorial_loop(n):
    result = 1
    for i in range(2, n + 1):
        result *= i
    return result
```

But let's sprinkle the magic dust of recursion:

```python
def factorial_recursive(n):
    if n == 1:
        return 1
    else:
        return n * factorial_recursive(n - 1)
```

This second snippet embraces the soul of recursion. We define an edge case — our base condition that stops the infinite tumble down the rabbit hole (n == 1). The function then keeps calling itself, shrinking the problem down each time (n - 1), till it hits that edge case. And then the unwinding begins, with each call resolving and returning its piece of the puzzle, eventually giving us our answer.

### Why Bother With Recursion?
Sure, recursion might seem like overkill for simple problems that a loop can fix. But consider the elegance of solutions it provides for more complex issues like tree traversals, algorithmic problem solving such as the Towers of Hanoi, or graph searches. Sometimes, recursion is not just a solution; it's the most intuitive one.

### Closing Thoughts
Once I got past the intimidating facade of recursion, I found a beautiful and streamlined way of solving certain kinds of problems. Just like my transition from being a REST advocate to appreciating the swiftness of gRPC, embracing recursion has opened up new pathways in my coding journey.

If you're still feeling the heebie-jeebies about recursion, you're not alone. Take it from someone who was once stumped by the concept: the enigma of recursion is worthy of exploration. With each recursive function we write, we're training our minds to see not just the stack of problems but the stack of solutions waiting beneath. Happy recursing!


Decoding Recursion: The Concept That Had Me Stumped


In this blog, we will explore the process of building a web scraper using HTML parsing capabilities in TypeScript. This is the first part of a two-part series, where we will use this HTML scraper to extract relevant information from a website and generate text files. In the second part of the series, we'll feed these files to a locally running GPT-4 instance, [privateGPT](https://github.com/imartinez/privateGPT), and analyze whether it can produce coherent code snippets based on the provided documentation.
For those eager to get their hands dirty, here's a link to the [repo](https://github.com/TylerMutai/coding-challenges-solutions/tree/main/lib/html-parsing-for-textual-output).

## Overview of the Web Scraper

Before diving into the code and its explanation, let's have a brief overview of our project. We aim to create a web scraper that can:

1. Download and parse HTML from a given target URL.
2. Extract all links from the web page.
3. Fetch content for each link and save it as a text file.
4. Calculate a similarity index between each text file and the overall content and ignore indices that already exist, or pieces of text that are at least 90% similar.

The project is built using TypeScript and consists of several modules that handle different functionalities. Let's analyze each module in chronological order, starting with the main entry point of our application – `html-web-parser.ts`.

## html-web-parser.ts

This is the main file of our web scraper which orchestrates the entire process. The scraper first fetches HTML content from the specified target URL. It then parses the content, extracts all hyperlinks from the page, and uses Puppeteer to fetch dynamic links generated by JavaScript.

Once URLs are collected, the scraper fetches the content of each link and saves it as a text file. For improved performance, we run this process using multiple worker threads (since our `getSimilarityIndex` function takes a while to complete, depending on the length of text), implemented in Node.js as `Worker`.

After saving content, the similarity index of each text file with the overall content is calculated and stored which we'll then use to gauge whether creating a new text file containing html content is necessary.

## lib/constants.ts

This is a simple module defining constants like the default output folder paths for our web scraper.

```typescript
export const DEFAULT_OUTPUT_FOLDER = "/tmp/html-web-parser/"
export const DEFAULT_OUTPUT_FOLDER_HTML = DEFAULT_OUTPUT_FOLDER + "html-output";
export const DEFAULT_OUTPUT_FOLDER_STORAGE = DEFAULT_OUTPUT_FOLDER + "storage";
export const DEFAULT_OUTPUT_FOLDER_LOCK = DEFAULT_OUTPUT_FOLDER + "lock";
```

## lib/lock.ts

`lock.ts` module provides a `Lock` class that deals with acquiring and releasing filesystem locks during concurrent read and write operations. We use this class to synchronize access to the number file in which similarity index values will be stored. This module also provides a `sleep` method to implement a waiting mechanism for locks and allow file referencing after the lock is released (lock file is deleted).

## lib/process-html-content.ts

This module contains a function `fetchAndSaveContent` responsible for fetching HTML content from the provided URL, extracting text content (textContent), and preparing it for further processing. It eliminates irrelevant links by using the `IGNORED_LINKS` array and focuses on links that belong to the same domain as the target URL. The function returns a tuple, with the first element being a unique identifier (filename) and the second element being the extracted text content.

## lib/save-html-content-to-file.ts

Using worker threads, the `save-html-content-to-file.js` module accepts workerData, containing the URL, output folder, text content, and overall content. In this module, the similarity index is calculated using `getSimilarityIndex()` function, and the text content is saved to a file named by the identifier provided by the `fetchAndSaveContent()` function.
The function `getSimilarityIndex` takes a long time to run, so we utilize nodejs threads to improve on execution speed.

## lib/storage.ts

The `Storage` class of `lib/storage.ts` deals with managing storage operations for similarity index values. It stores an in-memory set of similarity indexes and provides methods to lock/unlock file access to read and write the persistent storage securely such that all writes by the different running worker reads are all committed.

## lib/text-similarity-index.ts

This module provides a single function, `getSimilarityIndex()`, which calculates text similarity based on the edit distance (also known as Levenshtein distance). It measures the minimum number of edits required to transform one string into another.

## Building and Running the Web Scraper

After setting up a TypeScript configuration file, `tsconfig.json`, we compile the project using the following command: `tsc --project .`. Finally, we run the generated JavaScript file: `node output/html-web-parser.js https://riverpod.dev/docs/getting_started`. This command starts the web scraper, processing the given URL, and generating the text files.

Stay tuned for the second part of this series, where we will feed the generated files to privateGPT and observe whether it can produce coherent code based on the documentation.
If you have any thoughts, would like to improve different sections in this blog, or would just like to chat, feel free to reach out to me using any links on the [contact](https://www.balysnotes.com/contact) page.


Private GPT: Building a Web Scraper with HTML Parsing in TypeScript (Part 1).

Private GPT: Building a Web Scraper with HTML Parsing in TypeScript (Part 1)


GPT-4, despite its incredible linguistic prowess, suffers from a noteworthy constraint: its context length limitation, which essentially refers to the maximum number of input tokens. The sliding window technique, however, serves as a workaround by breaking up a document into overlapping segments that GPT-4 can easily digest.

In this blog post, we'll examine the implementation of the sliding window technique to bypass GPT-4's context length restriction, discuss associated trade-offs, and identify optimal chunk and overlap lengths for best results.

### Harnessing the Sliding Window Technique
The sliding window technique entails dividing a document into smaller segments within GPT-4's context length limit, processing each segment, and then reassembling them. The primary challenge when employing this method is preserving context and fluidity among processed segments.

To accomplish this, we establish a chunk length and an overlap length that guarantee context continuity. The chunk length must be equal to or less than GPT-4's context length limit, while the overlap length should be adequately large to facilitate contextual comprehension between processed segments.

For instance, if GPT-4's context length limit is 2048 tokens, we might use a chunk length of 2000 tokens and an overlap length of 100 tokens. This way, when stitching processed segments back together, the overlapping sections help maintain context and minimize duplication.
Here's a TypeScript example illustrating how to implement this. Note that the `gpt4Function` should be your helper function responsible for making API calls to OpenAI.

```typescript
async function processTextFile(filePath: string, chunkLength: number, overlapLength: number): Promise<void> {
  const readInterface = readline.createInterface({
    input: fs.createReadStream(filePath),
    output: process.stdout,
    terminal: false
  });

  let buffer = '';
  let result = '';
  let isFirstChunk = true;
  const outputFile = filePath.replace('.txt', '_new.txt');

  for await (const line of readInterface) {
    buffer += ' ' + line;

    while (buffer.length >= chunkLength) {
      let chunk = buffer.slice(0, chunkLength);
      buffer = buffer.slice(chunkLength - overlapLength);

      let processedChunk = await gpt4Function(chunk);
      if (isFirstChunk) {
        result += processedChunk;
        isFirstChunk = false;
      } else {
        result += processedChunk.slice(overlapLength);
      }
    }

    fs.writeFileSync(outputFile, result, {flag: 'w'});
  }

  if (buffer.length > 0) {
    let lastProcessedChunk = await gpt4Function(buffer);
    result += lastProcessedChunk.slice(overlapLength);
  }

  fs.writeFileSync(outputFile, result, {flag: 'w'});
}
```
### Selecting the Ideal Chunk Length and Overlap Length
Striking the perfect balance between chunk length and overlap length is key to achieving optimal results. Begin by setting the chunk length close to GPT-4's context length limit and determining a reasonable overlap length based on empirical testing (trial and error).

### Applications
1. Summarizing Large Documents: The sliding window technique makes summarizing extensive articles much more manageable, delivering coherent and precise summaries to users.
2. Proofreading and Grammar Correction: Massive text files can be swiftly checked for grammar mistakes and punctuation problems.
3. Plagiarism Detection: By utilizing the sliding window technique to process large volumes of text, plagiarism in academic or corporate environments can be detected.

### Limitations
While the sliding window technique is undeniably useful for processing large documents and generating consistent and accurate input, it does come with certain drawbacks.
For example, when summarizing large documents, small overlaps may be insufficient for preserving context in specific cases, potentially resulting in a loss of meaning or continuity.
Furthermore, this method increases usage costs. OpenAI charges per API call, so making multiple calls for different chunks will drive up costs.

### Conclusion
Although we continue to grapple with GPT-4's context length limitation, the sliding window technique allows us to exploit GPT-4's full potential across a range of applications such as summarization, proofreading, and plagiarism detection. The next GPT-4 iteration will feature a much larger context window which will help circumvent some limitations of using this technique.


Overcoming GPT-4's Context Length Limitation with the Sliding Window Technique

Bypassing GPT-4's Context Length Limitation with Sliding Window Technique


Welcome back, fellow video conferencing enthusiasts! In the previous parts of our blog series, we've covered accessing the webcam's video stream and setting up a signalling server. Now it's time for the grand finale: setting up the client to stream their video and receive video from other clients. Let's dive in!

Here's [part 1](https://www.balysnotes.com/setting-up-a-peer-to-peer-connection-video-stream) and [part 2](https://www.balysnotes.com/setting-up-a-signaling-server-client-video-streaming).

Here's the link to the [repo](https://github.com/TylerMutai/peer-to-peer-video-conferencing).

Our HTML file includes a local video element and a remoteVideos div where we'll add video elements for each connected peer. We're also including the socket.io library and a script that contains the main logic.

```html
<video class="video" id="localVideo" autoplay playsinline muted></video>
<div id="remoteVideos"></div>
<script src="/socket.io/socket.io.js"></script>
```

We start off by defining some constants and an empty object called `peers` to store the PeerConnections. We'll also need a function called `createPeerConnection` that takes the sender's ID and an ontrack event handler as arguments. This function sets up a new RTCPeerConnection with the specified configuration, including the Google STUN server.

```javascript
const socket = io();
const roomId = 'test-room';
const localVideo = document.getElementById('localVideo');
const remoteVideos = document.getElementById('remoteVideos');
const configuration = {iceServers: [{urls: 'stun:stun.l.google.com:19302'}]};
const peers = {};
```

The `createPeerConnection` function sets up the icecandidate and ontrack event listeners for the connection. It then returns the new connection.

```javascript
function createPeerConnection(senderId, ontrack) {
  const pc = new RTCPeerConnection(configuration);

  pc.onicecandidate = (event) => {
    if (event.candidate) {
      socket.emit('icecandidate', {receiverId: senderId, candidate: event.candidate});
    }
  };

  pc.ontrack = ontrack;

  return pc;
}
```

Next up, we have the `createOffer` function. This will create an offer for the specified sender ID. We first get the user media stream and set the local video source. We then add the stream's tracks to the PeerConnection, create an offer, and set the local description. Finally, we send the offer to the receiver through the signalling server.

```javascript
function createOffer(senderId) {
  const pc = peers[senderId];
  navigator.mediaDevices.getUserMedia({video: true, audio: true})
    .then((stream) => {
      localVideo.srcObject = stream;
      stream.getTracks().forEach(track => pc.addTrack(track, stream));

      pc.createOffer()
        .then(offer => pc.setLocalDescription(offer))
        .then(() => {
          socket.emit('offer', {receiverId: senderId, offer: pc.localDescription});
        });
    });
}
```

Now, let's wire up our socket event listeners. When a new peer connects, we create a new video element for them, add it to the remoteVideos div, and set up a new PeerConnection. Then, we call the `createOffer` function for that peer.

```javascript
socket.on('peer-connected', (data) => {
  const clientId = data.clientId;
  const remoteVideo = document.createElement('video');
  remoteVideo.id = `remoteVideo_${clientId}`;
  remoteVideo.classList.add("video")
  remoteVideo.autoplay = true;
  remoteVideo.playsInline = true;
  remoteVideos.appendChild(remoteVideo);

  peers[clientId] = createPeerConnection(clientId, (event) => {
    remoteVideo.srcObject = event.streams[0];
  });
  createOffer(clientId);
});
```

On receiving an offer, we set the remote description of the PeerConnection and create an answer. We then send the answer back to the sender through the signalling server.

```javascript
socket.on('offer', async (data) => {
  const senderId = data.senderId;
  const pc = peers[senderId];
  await pc.setRemoteDescription(new RTCSessionDescription(data.offer));
  const answer = await pc.createAnswer();
  await pc.setLocalDescription(answer);
  socket.emit('answer', {receiverId: senderId, answer});
});
```

When we receive an answer, we set the remote description of the PeerConnection.

```javascript
socket.on('answer', async (data) => {
  const senderId = data.senderId;
  const pc = peers[senderId];
  if (pc) {
    await pc.setRemoteDescription(new RTCSessionDescription(data.answer));
  }
});
```

On receiving an icecandidate event, we add the candidate to the PeerConnection.

```javascript
socket.on('icecandidate', (data) => {
  const senderId = data.senderId;
  const pc = peers[senderId];
  if (pc) {
    pc.addIceCandidate(new RTCIceCandidate(data.candidate));
  }
});
```

Finally, on a peer-disconnected event, we remove the video element and close the PeerConnection.

```javascript
socket.on('peer-disconnected', (data) => {
  const clientId = data.clientId;
  const videoElement = document.getElementById(`remoteVideo_${clientId}`);
  if (videoElement) {
    videoElement.remove();
  }
  if (peers[clientId]) {
    peers[clientId].close();
    delete peers[clientId];
  }
});
```

To see this in action, within your `package.json`, include this piece of code if it's not there already:

```json
{
  "scripts": {
    "start": "ts-node server.ts"
  }
}
```

lastly, run `npm start` and open tabs in different profiles (i.e. guest mode and incognito mode) to see the different video streams.


And there you have it! Our video conferencing masterpiece is complete. Give yourself a pat on the back. With this setup, you can now stream video and receive video from other clients using WebRTC and socket.io.


The Final Act: Streaming Video and Receiving it in a WebRTC Video Conference (Part 3/3)


Welcome back to our thrilling three-part series on setting up a peer-to-peer connection for client video streaming. In [Part 1](https://www.balysnotes.com/setting-up-a-peer-to-peer-connection-video-stream), we got our hands dirty by setting up the basics of a client-side WebRTC application. Now, it's time to tackle the server-side of things. Buckle up, because we're about to dive into the fascinating world of signalling servers and WebRTC peer information exchange!

Here's the link to the [repo](https://github.com/TylerMutai/peer-to-peer-video-conferencing).

## Previously on "Client Video Streaming"...

In the last episode, we set up our client-side application to establish a WebRTC connection, allowing video streaming between two clients. However, our two clients were lonely and couldn't find each other in the vast internet space. This is where our signalling server comes to the rescue. It will act as a matchmaker, helping the two clients find each other and exchange the necessary information to start their video streaming relationship.

## A Signalling Server to the Rescue

Now that we've set the stage, let's get down to business. Our mission today is to set up a signalling server that will help users find one another and establish a WebRTC connection. 
As a reference, we'll be using the code snippet provided below. We'll be using TypeScript, Express, and Socket.IO to make this happen.

```typescript
import express from 'express';
import { Server as HttpServer } from 'http';
import { Server as SocketIOServer, Socket } from 'socket.io';

const app = express();
const httpServer = new HttpServer(app);
const io = new SocketIOServer(httpServer);

app.use(express.static('public'));

io.on('connection', (socket: Socket) => {
  console.log('User connected:', socket.id);

  socket.on('join-room', (roomId: string) => {
    console.log('User joined room:', roomId);
    const roomClients = io.sockets.adapter.rooms.get(roomId);

    if (!roomClients || roomClients.size === 0) {
      socket.join(roomId);
    } else {
      socket.emit('room-full', 'Room is full');
      socket.disconnect();
    }
  });

  socket.on('offer', (data: { roomId: string; offer: RTCSessionDescriptionInit }) => {
    socket.to(data.roomId).emit('offer', data.offer);
  });

  socket.on('answer', (data: { roomId: string; answer: RTCSessionDescriptionInit }) => {
    socket.to(data.roomId).emit('answer', data.answer);
  });

  socket.on('icecandidate', (data: { roomId: string; candidate: RTCIceCandidateInit }) => {
    socket.to(data.roomId).emit('icecandidate', data.candidate);
  });

  socket.on('disconnect', () => {
    console.log('User disconnected:', socket.id);
  });
});

const PORT = process.env.PORT || 3000;
httpServer.listen(PORT, () => {
  console.log(`Server is running on port ${PORT}`);
});
```

### Step 1: Setting up the server

First things first, let's set up our server using Express and Socket.IO. Our server will be listening on port 3000 (or any available port in the environment). Here's the code to get our server up and running:

```typescript
import express from 'express';
import { Server as HttpServer } from 'http';
import { Server as SocketIOServer, Socket } from 'socket.io';

const app = express();
const httpServer = new HttpServer(app);
const io = new SocketIOServer(httpServer);

const PORT = process.env.PORT || 3000;
httpServer.listen(PORT, () => {
  console.log(`Server is running on port ${PORT}`);
});
```

### Step 2: Serving static files

To serve our client-side files, we'll use the `express.static` middleware. This will enable our server to serve static files from the `public` folder:

```typescript
app.use(express.static('public'));
```

### Step 3: Handling Socket.IO connections

Now, it's time to handle the connections coming from our clients. We'll listen for the `connection` event on our Socket.IO server and print a message whenever a user connects:

```typescript
io.on('connection', (socket: Socket) => {
  console.log('User connected:', socket.id);
});
```

### Step 4: Room management and WebRTC signalling

Our server will be responsible for managing rooms and exchanging WebRTC peer information between the clients. To do this, we'll listen for various events, such as `join-room`, `offer`, `answer`, and `icecandidate`. We'll also handle the `disconnect` event to clean up after a user leaves:

```typescript
socket.on('join-room', (roomId: string) => {
  // ...
});

socket.on('offer', (data: { roomId: string; offer: RTCSessionDescriptionInit }) => {
  // ...
});

socket.on('answer', (data: { roomId: string; answer: RTCSessionDescriptionInit }) => {
  // ...
});

socket.on('icecandidate', (data: { roomId: string; candidate: RTCIceCandidateInit }) => {
  // ...
});

socket.on('disconnect', () => {
  console.log('User disconnected:', socket.id);
});
```

Within each event handler, we'll manage the rooms and forward the WebRTC information to the appropriate clients. For example, when handling the `join-room` event, we'll check if the room is full and either let the user join or disconnect them:

```typescript
socket.on('join-room', (roomId: string) => {
  console.log('User joined room:', roomId);
  const roomClients = io.sockets.adapter.rooms.get(roomId);

  if (!roomClients || roomClients.size === 0) {
    socket.join(roomId);
  } else {
    socket.emit('room-full', 'Room is full');
    socket.disconnect();
  }
});
```

And there you have it! With these event handlers in place, our signalling server is now ready to help clients exchange WebRTC peer information and establish video streaming connections.

## Coming Soon: The Final Chapter

Phew! Our signalling server is up and running, and our clients are no longer lonely. They can now find each other and start streaming video, all thanks to our trusty server-side matchmaker. But our journey doesn't end here! In the next and final part of this series, we'll put everything together and make some final adjustments to our application, ensuring that our video streams seamlessly between clients.

See you in the last part of this series!


Setting Up a Signalling Server: Client Video Streaming (Part 2 of 3)


Welcome to the first part of our three-part series on setting up a peer-to-peer (P2P) connection to send a video stream
between two clients (a server serving as an intermediary only for initial connection set-up).
We'll demonstrate this plain Javascript, HTML and CSS. Strap in, it's going to be a fun ride!

Here's the link to the [repo](https://github.com/TylerMutai/peer-to-peer-video-conferencing).

In this first installment, we'll be talking about the available Web APIs for obtaining a video stream from the client.
To get started, let's take a look at a piece of JavaScript code that we'll be using as a reference:

```typescript
const getWebcamStream = async () => {
  const videoRef = document.getElementById('localVideo')
  try {
    const videoStream = await navigator.mediaDevices.getUserMedia({
      video: {
        facingMode: facingMode.current
      }
    });
    if (videoStream && videoRef.current) {
      // set this video stream to our video stream object.
      videoRef.srcObject = videoStream
    }
  } catch (e: any) {
    console.log(e?.message || "Could not start video stream. Have you enabled permissions?")
  }
}
```

Now, let's dive into the nitty-gritty details of this code and see what's happening under the hood.

### Web APIs: Browser Support for Video Streaming

The code snippet above showcases a few Web APIs that work together to obtain a video stream from the client's device.
Let's break them down one by one:

#### `navigator.mediaDevices.getUserMedia`

`getUserMedia` is a method available through the `navigator.mediaDevices` object, and it returns a Promise that resolves
to the requested `MediaStream` object containing live audio and/or video streams. In our code snippet, we use
the `await` keyword to fetch the video stream asynchronously.

The `getUserMedia` method accepts an object detailing the media constraints you want to apply to your stream. In our
case, we're only interested in obtaining a video stream, so we pass `{ video: { facingMode: facingMode.current } }`.
This tells the method to grab a video stream with the specified facing mode (either `user` for the front camera
or `environment` for the rear camera on mobile devices).

#### `videoRef.current.srcObject`

Once we have the video stream, we need to display it on our web page. To do this, we set the `srcObject` property of our
video element (`videoRef.current`) to the obtained video stream. Now the video element knows which stream to play, and
the user can see what their camera is capturing.

### Stopping the Video Stream

Before we wrap up, let's take a look at another piece of JavaScript code that handles stopping the video stream:

```typescript
const handleStopStream = async () => {
  const videoRef = document.getElementById('localVideo')
  const stream = videoRef.srcObject
  if (stream) {
    if ("getTracks" in stream) {
      const tracks = stream.getTracks();
      for (let i = 0; i < tracks.length; i++) {
        tracks[i].stop();
      }
      videoRef.srcObject = null;
    }
  }
}
```

This code snippet introduces a function called `handleStopStream` that stops the video stream when necessary. Let's
break it down:

#### `videoRef.srcObject`

First, we check if there's an active video stream by accessing the `srcObject` property of the video
element (`videoRef`). If a stream exists, we proceed to stop it.

#### `stream.getTracks` and `tracks[i].stop()`

As we saw earlier, `getTracks` returns an array of `MediaStreamTrack` objects representing the audio and video tracks in
the stream. To stop the stream, we loop through these tracks and call the `stop` method on each one. This effectively
stops the video stream from capturing data and frees up resources.

#### `videoRef.srcObject = null`

Once the tracks are stopped, we set the `srcObject` property of the video element to `null`. This removes the video
stream from the video element and stops displaying the video on the web page.

### Wrapping Up

So there you have it! With just a few lines of code and some powerful Web APIs, we can obtain a video stream from the
client's device, display it on a web page, and set the stage for sharing it with other clients using P2P connections.

Stay tuned for the next part of this series, where we'll explore how to set up WebRTC using Socket.io.
Until then, happy coding! 


Setting Up a Peer-to-Peer Connection: Client Video Streaming (Part 1 of 3)


In today's world, microservices architecture has become a popular approach for building scalable and reliable applications. With microservices, an application is broken down into smaller, independent services that can be developed and deployed separately. This approach then necessitates the need for a communication protocol that will allow these services to talk to each other. Traditionally, REST APIs have been used to serve this purpose, but there's a new kid on the block with a claim to the throne called gRPC. gRPC works like normal RPC calls, allowing you to call methods implemented on other machines directly.

In this blog post, we'll be looking into the benefits of gRPC and how to implement server side code using node and typescript. We'll be using Postman as a client for calling and testing our gRPC method implementations. Here's a link to [download it](https://www.postman.com/downloads/).

### A Brief Introduction to gRPC

gRPC is a high-performance, open-source, universal RPC (Remote Procedure Call) framework developed by Google. It is built on top of the newer HTTP/2 protocol and uses Protocol Buffers (rather than strings/text), making it efficient, lightweight, and easy to use. This also means faster and more efficient communication between services when compared to traditional REST APIs.

Protocol Buffers (that we'll also call protobuf in this blog post) are a language-agnostic mechanism for serializing structured data. The data format is in binary and not text (as is in traditional REST APIs that use JSON) and therefore occupies a smaller memory footprint making it suitable even for devices with low memory like embedded systems.
Similarly, the gRPC framework ships with the protocol buffer compiler which generates your data classes for you (from defined protofiles, which we'll cover in this blog post) reducing the amount of boilerplate code that needs to be written.

As with all other technologies, gRPC does have its fair share of disadvantages. For starters, the learning curve, when compared to traditional JSON APIs, is rather steep and might take some time for newer developers to be fully acquainted. gRPC also uses the more modern HTTP/2 protocol which has limited browser support, therefore adding set-up overhead and the need to use proxy servers to interface between HTTP/2 and HTTP/1 protocols. Your choice to use gRPC in your setup therefore boils down to your specific use case. A good place to start is for communication between internal microservices where you won't need to build and support client libraries.

### The Implementation in Node and TypeScript
We'll do a simple server set up that will allow users to login and perform CRUD operations on their contacts.

One key difference between gRPC and REST APIs is that gRPC defines its own standard for status codes which differ from the conventional HTTP response codes that most of us are farmiliar with. Here's a comprehensive [list](https://grpc.github.io/grpc/core/md_doc_statuscodes.html) of all these status codes. As such, in each of our protofile definitions, in the responses message types, we'll include a field called `status` that we'll use to store the standard HTTP status codes.

For those who can't wait to get their hands dirty and want to dive directly into the source code, here's the codebase link: [gRPC Server Example](https://github.com/TylerMutai/grpc-server-example.git). For those of us who are mere mortals and prefer a more surface-level understanding, keep on reading.

#### Prerequisites
To generate data classes from your proto definitions, you'll need to have the Protocol Buffer Compiler installed. Heres a handy [link](https://grpc.io/docs/protoc-installation/) on how you can go about it. If you get 'a protoc-gen-ts program not found or is not executable' error, you'll need to downgrade protobuf. Here's another handy [link](https://stackoverflow.com/questions/72572040/protoc-gen-js-program-not-found-or-is-not-executable) on how to do this.

You also need to have node installed. [Here's the download link](https://nodejs.org/en/download).

#### Project Set-Up
Our folder structure will be as follows:
root
|\___ __src__ (houses our project source files)
|\____ __protos__ (where we'll define our protofiles and where protoc will generate revelant ts files)
|\____ __services__ (where we'll define the server side implementation of the services we've defined in our protofiles)
|\____ __types__ (where we define necessary interface types for different data structures)
|\____ __utils__ (where we'll define utility functions that are re-usable globally across the project)
|\____ __values__ (where we define global constants. i.e. server host etc.)
|\____ __index.ts__ (the entry point of our application. Handles init functions like server startup and binding to the defined port)

Feel free to create a nodejs project at a location of your choice (with typescript support) that mimicks this folder strucutre.
We'll then create a file named `MakeFile` (with no extension) at the root of the project you just created, then copy paste the following:

```bash
PROTO_DIR=src/protos

proto_types:
	grpc_tools_node_protoc \
	--js_out=import_style=commonjs,binary:${PROTO_DIR} \
	--grpc_out=grpc_js:${PROTO_DIR} \
	--plugin=protoc-gen-grpc=`which grpc_tools_node_protoc_plugin` \
	-I ${PROTO_DIR} \
	${PROTO_DIR}/*.proto

proto_web:
	protoc \
	--plugin=protoc-gen-ts=./node_modules/.bin/protoc-gen-ts \
	--ts_out=grpc_js:${PROTO_DIR} \
	-I ${PROTO_DIR} \
	${PROTO_DIR}/*.proto

proto: proto_types proto_web
```
What this file does is basically house our collection of bash commands related to the protoc compiler that we can conveniently execute by running `make proto` on the terminal whenever we want to generate service implementations from our `.proto` files.

#### Protofile Definitions
The first step is to define the protocol buffer (.proto) files, which define the services and messages that will be used for communication between services. In this example, we have two .proto files, one for authentication and another for contacts.
We'll use a simple implementation of the JWT flow for user authentication. If you aren't farmiliar with JWTs, I'd suggest you take a read before exploring the codebase as you may get lost in authentication technicalities, which is not what this blog post is trying to explore.

Our authentication service (at `src/protos/auth.proto`) is defined as follows:

auth.proto:
```protobuf
syntax = "proto3";

package auth;

service AuthService {
  rpc Login (LoginRequest) returns (LoginResponse);
  rpc UserMe (UserRequest) returns  (UserResponse);
  rpc RefreshAccessToken (RefreshAccessTokenRequest) returns  (RefreshAccessTokenResponse);
}

message UserRequest{}

message UserResponse {
  int32 status = 1;
  string error = 2;
  string username = 4;
  string id = 5;
  string password = 6;
}

message LoginRequest{
  string username = 1;
  string password = 2;
}

message LoginResponse{
  int32 status = 1;
  string error = 2;
  string jwtToken = 3;
  string refreshToken = 4;
}

message RefreshAccessTokenRequest{
  string refreshToken = 1;
}

message RefreshAccessTokenResponse{
  int32 status = 1;
  string error = 2;
  string accessToken = 3;
}
```
Looking at the code snippet above, defining a protofile is fairly straightforward. You start off by defining a `message` that represents a data structure. Within the `message` definitions, you define your various fields and their respective data type (for those coming from a dynamically typed language, a data type is basically a data structure. I.e. an Array is a data type. A dictionary is also a data type. This also applies to integers, floating point numbers and strings, which are also data types).

In this case, we've defined a `AuthService` message that has three methods:
1. `Login`- Takes in a `username` and `password` as inputs and responds with an access token and refresh token.
2. `UserMe` - Takes no inputs (only the access token as metadata) and responds with information related to the logged in user.
3. `RefreshAccessToken` -  Takes a refresh token as input (generated during login) and responds with a new access token.

Our contacts service protofile also follows a similar pattern (at `src/protos/contacts.proto`). Feel free to explore this in the example repository.

Lastly, we run `make proto` from the root of the project to generate our typescript data classes with the necessary boilerplate code that we'll interact with when doing our server implementation.

#### Server Implementation
Here, we'll define the logic based on each of the method messages we defined earlier in our protofiles. We'll start off with our `AuthService`. Kindly note that I'll be using 'method' and 'function' here interchangeably, but they both refer to typescript functions.
Also, feel free to download the `package.json` file from the repository and run `npm install` to load up the packages that we'll be using in this implementation.

We'll walk through different sections of our service implementation (at `src/services/authService.ts`) as I attempt to explain the structure of this file, and how we've done the implementation.

As is the norm in I believe every programming language, we start off by importing the necessary files that will help us wire our service.

```typescript
import {sendUnaryData, ServerUnaryCall} from "@grpc/grpc-js";
import {generateAccessToken, generateRefreshToken, getLoggedInUser, refreshToken} from "../utils/auth";
import {User} from "../types/user";
import {
    LoginRequest,
    LoginResponse,
    RefreshAccessTokenRequest,
    RefreshAccessTokenResponse,
    UserRequest,
    UserResponse
} from "../protos/auth_pb";
```
Here, we import core classes defined in the `@grpc/grpc-js` package that define gRPC related core classes. In our case, we'll only handle unary server calls (gRPC also supports streaming calls, which we'll cover in a future post).

We then import typescript definitions of the protofiles we declared above (in our `auth.proto`) that were generated by the `make proto` command.


We then define mock user data that we'll use during login since we won't be handling any user registration logic.

```typescript
// Mock user data for authentication
const USERS = new Map<string, User>();
USERS.set(
  "admin", {
    id: 1, username: "admin", password: "admin",
  }
)
USERS.set(
  "staff", {
    id: 2, username: "staff", password: "staff",
  }
)

```

All method implementations will follow a similar pattern. They all have 2 parameters: a `call` parameter of type `ServerUnaryCall` and a `callback` parameter of type `sendUnaryData`. Both types take generic arguments.
Our `ServerUnaryCall` takes 2 generic arguments: the first one for the request definition and the other for the response definition. I'll therefore only explore the method `userMe` to talk about how we implement the various methods defined in our protofiles.
Again, feel free to explore the whole file in the example repository.

```typescript
export const userMe = (call: ServerUnaryCall<UserRequest, UserResponse>, callback: sendUnaryData<UserResponse>) => {
  const response = new UserResponse();
  const user = getLoggedInUser();
  response.setId(`${user.id}`);
  response.setStatus(200);
  response.setError("");
  response.setUsername(user.username);
  response.setPassword(user.password);
  callback(null, response)
}
```
When doing method implementations, our method names should correspond to the method names we defined in our protofiles. This is not a hard constraint, but is generally better since it's clear at first glance which method here corresponds to which method in the defined protofiles. In the case above, `userMe` corresponds to `UserMe` defined in the `auth.proto` protofile.

The `call` parameter in our method takes a `ServerUnaryCall` with `UserRequest` and `UserResponse` as generic arguments. The `UserRequest` and `UserResponse` types are imported from `src/protos/auth_pb.ts` which is part of the files that were generated by the protoc compiler when we ran `make proto`. These class definitions directly correspond to your protofile message definitions all the way down to the data types. The protoc compiler also includes helper methods like setters and getters when generating source files that will allow you to manipulate values conveniently.

The `callback` parameter takes a `sendUnaryData` type with only one generic argument. The generic argument represents the type of response we'll be returning, in this case `UserResponse`. The `callback` function also takes an initial parameter (that we have passed `null` as) which represents an error - if any - of  type `ServerErrorResponse`. You can use this parameter to throw gRPC specific errors i.e. when a user is not authenticated.

Inside the method definition, we declare a `response` variable and instantiate it with the `UserResponse` class. We then handle necessary logic and set the status values accordingly. We defined the `status` variable in our protofiles to store standard HTTP status codes i.e. 200 OK since gRPC defines its own standard set of status codes. If we wanted to get any values attached in the incoming request, we can as well get a reference to this by invoking `call.request`.

We lasty invoke the callback function with our response object and errors (if any).

That's pretty much all it takes to define and implement server logic in gRPC.

#### Server Set-Up
Lastly, we need to start our server and attach the our service implementations to this server instance. This will be done in the `index.ts` file at the root of our project. Here, we'll also be covering a concept called `interceptors` which will allow us to intercept our requests and responses and handle logic before they are passed to our services or before responses are returned to client. In our case, we'll use interceptors to check whether a client is already authenticated with our backend, and throw an error if not and the client is trying to invoke a method that requires authentication. We'll use the interceptor implementation from the package `grpcjs-interceptors`.

```typescript
const interceptors = require('grpcjs-interceptors');

const server = interceptors.serverProxy(new Server());

server.addService(ContactServiceService, {
  addContact,
  getContacts,
  updateContact,
  deleteContact
});

server.addService(AuthServiceService, {
  login,
  userMe,
  refreshAccessToken
});
```
In this code snippet, we instantiate our server through the `grpcjs-interceptors` package so that we can attach global interceptors to our server. We then add the services we just implemented to our server instance, in this case, the auth and contacts services.

```typescript
const openEndpoints = [
  '/auth.AuthService/Login',
  '/auth.AuthService/RefreshAccessToken'
]
const checkAuthorizationToken = async function (ctx, next, callback) {
  if (!openEndpoints.includes(ctx.service.path)) {
    // check if user is authorized to access this route.
    const metadata = ctx.call.metadata;
    const authToken = metadata.get("authorization").toString();
    const userIsAuthenticated = await isAuthenticated(authToken)
    if (!userIsAuthenticated) {
      callback(new Error("Unauthorized."))
      return;
    }
  }
  await next()

  // do something before response goes back to client
}
```
We then define a variable `openEndpoints` which will store the method definitions that do not require authentication. The strings defined here follow a specific syntax. You start off with the package name defined in your protofile i.e. `auth`, then a forward slash(`/`) followed by the service name as defined in your protofile i.e. `AuthService` followed by a forward slash (`/`) and lastly the method name, again as defined in your protofile i.e. `Login`. This structure is defined by the gRPC standard and you can see it when you console.log a request's metadata, in this case: `console.log(ctx.service.path)`.
Afterwards, we define our interceptor function which will handle logic for ensuring that only the closed endpoints are accessible by authenticated users. Here, we've defined an `Error` object that will fire using gRPC specific status codes in case a user is not authenticated. We then cancel that request and respond with that error.
If you'd like to perform a function after the request has been processed, you should instead handle logic after the `await next()` block.

```typescript
server.use(checkAuthorizationToken);

server.bindAsync(host, ServerCredentials.createInsecure(), (err, port) => {
  if (err) {
    console.log(err)
    return;
  }
  server.start();
  console.log(`listening on ${host}`)
});
```
Lastly, we tell our server to use the interceptor we've just defined. You can attach more than one interceptor to your server by using the `server.use(interceptor)` function.
We then bind our server to the host and port defined in `src/values/globals.ts`.

We can now test our server implementation using postman. Here's the [documentation](https://learning.postman.com/docs/sending-requests/grpc/grpc-request-interface/) on how to do that.

### Conclusion
gRPC is a powerful and efficient communication protocol that is well-suited for modern microservices architecture. It offers several benefits, including high performance, language-independence, interoperability, code generation, and streaming. With official support for Node.js and TypeScript, it is easy to use in modern web applications. Although the use of interceptors may seem complex at first glance, they are a powerful tool for intercepting and modifying requests and responses, and can be used to implement complex authentication and authorization logic.

Feel free to share your thoughts!



gRPC: A Modern Approach to Microservices Communication


Hello, fellow code connoisseurs! I've recently decided to refresh my memory on data structures and algorithms. This current project is inspired by freecodecamp's tic-tac-toe example. Buckle up, and let's get started!

This code snippet revolves around a Tic-Tac-Toe game but with some added features, such as the ability to change the board size and randomize the starting player. The core functionality of the game is handled by a reducer called `interactionAreaReducer`, which takes in the current state and an action and returns the updated state based on the action.

We'll only be exploring this file (as this is where the core logic of the game is handled). Feel free to explore the rest of the repository here:  [tic-tac-toe-example](https://github.com/TylerMutai/tic-tac-toe-example.git).

Here's also a reference to [React reducers](https://react.dev/learn/extracting-state-logic-into-a-reducer) if you aren't familiar.

### Let's Get Started!

#### 1. Importing essential modules and types

This piece of art begins by importing various modules and types, such as `InteractionAreaState`, `playerTypes`, `InteractionAreaAction`, and `toTitleCase`. I'll explain each briefly.

```typescript
import InteractionAreaState from "@/reducers/interactionArea/state";
import playerTypes from "@/types/playerTypes";
import {InteractionAreaAction} from "@/reducers/interactionArea/action";
import {toTitleCase} from "@/utils/general";
import PlayerTypes from "@/types/playerTypes";
```

1. `InteractionAreaState` - An interface describing what our interaction area will have. 'InteractionArea' here
   referes to the part of the board a user interacts with when playing a move.
2. `PlayerTypes` - Note the uppercase 'P'. Refers to the different types of players. In this case, I haven't defined a 'computer' type and is therefore by default 2 player mode ('player-1' and 'player-2').
3. `playerTypes` - Note the lowercase 'p'. Refers to an array of type 'PlayerTypes'.
4. `InteractionAreaAction` - Refers to the different types of actions you can perform on the interaction area. This maybe be: `'reset_game'`, `'start_game'` etc. They have been defined in the repo if you'd like to explore further.
5. `toTitleCase` - Just a utility function that uppercases each first letter of every word in a sentence.

#### 2. Defining the initial state
The `interactionAreaInitialState` object sets up the game's initial state.

```typescript
export const interactionAreaInitialState: InteractionAreaState = {
  playedMoves: new Set<string>(),
  cellsPlayed: new Map<string, playerTypes>(),
  boardSize: 3,
  hasGameStarted: false,
}
```
`playedMoves` here is a set since we want constant time lookup when a move is made so as to improve performance. Our board (as defined by `boardSize`) can grow to any size and therefore this performance difference becomes noticable if, say we were to use an array, as the board size increases.
`cellsPlayed` is a map that stores the coordinates each player has played. i.e. if a player clicks on the coordinate '[2][1]', we save a key of '21' and the value becomes the player i.e. 'player-1'. We use this map when doing a lookup of whether the game has a winner, and who the winner actually is.

#### 3. Crafting the reducer:
The `interactionAreaReducer` is a function that takes the current state and an action, then returns the updated state based on the action type.  This is where the bulk of our logic for starting a game, checking for a winner, checking for a draw, checking for invalid moves and resetting the game. We'll go through our switch statement step by step.
```typescript
const interactionAreaReducer = (state: InteractionAreaState, action: InteractionAreaAction): InteractionAreaState => {
  switch (action.type) {
    case "start_game":
      // randomize between 'computer' and 'player-1'. 'player-1' will be '1' and 'computer will be '2'
      const start = Math.round((Math.random() * 2));
      return {...state, hasGameStarted: true, currentPlayer: start === 1 ? "player-1" : "player-2"}
}

```
Here, we randomly select which player starts. We round off the number to the nearest whole number since generating random numbers returns a floating point number.

```typescript
const interactionAreaReducer = (state: InteractionAreaState, action: InteractionAreaAction): InteractionAreaState => {
  switch (action.type) {
    case "show_reset_button":
      return {...state, shouldShowResetButton: true}
}

```
Here, when all moves are exhausted, or a winner has been found, this action is invoked to show the button that when clicked will reset state.
```typescript
const interactionAreaReducer = (state: InteractionAreaState, action: InteractionAreaAction): InteractionAreaState => {
  switch (action.type) {
    case "hide_reset_button":
      return {...state, shouldShowResetButton: false}
}

```
This action here hides the reset button. i.e. when a new game has started.

```typescript
const interactionAreaReducer = (state: InteractionAreaState, action: InteractionAreaAction): InteractionAreaState => {
  switch (action.type) {
      case "reset_game":
      return interactionAreaInitialState;
}

```
This action returns the initial state, basically resetting state.

```typescript
   case "play_move":
      // check if the move being played is possible
      const player = action.payload?.player || "player-1"; // Can never be undefined
      const move = action.payload?.move || ""; // Can never be undefined
      const playedMoves = new Set(state.playedMoves);
      const cellsPlayed = new Map(state.cellsPlayed);
      const boardSize = state.boardSize;
     
      // 1. Check if this move is available. If not, throw error.
      if (playedMoves.has(`${move[0]}${move[1]}`)) {
        alert("This move is not available");
        return {...state};
      }

      // 2. Inject the current move (Play the move)
      playedMoves.add(move);
      cellsPlayed.set(move, player);


      // 3. Check for winner
      if (checkWinner(boardSize, cellsPlayed)) {
        return {...state, shouldShowResetButton: true, playedMoves, cellsPlayed}
      }

      // 4. If no winner was found, and there aren't any other moves, end game.
      if (cellsPlayed.size === (boardSize * boardSize)) {
        // No more moves to play
        alert(`Game over. No player has won. Click on the reset button to start over.`);
        return {...state, shouldShowResetButton: true, playedMoves, cellsPlayed}
      }

      // 3. Play the move.
      const newCurrentPlayer = player === "player-1" ? "player-2" : "player-1";
      return {...state, playedMoves, cellsPlayed, currentPlayer: newCurrentPlayer}
```
Here, I think the comments are self explanatory. we'll switch over to the `checkWinner` function as this is the function that handles logic for selecting a winner.

```typescript
const checkWinner = (boardSize: number, cellsPlayed: Map<string, playerTypes>): boolean => {

```
This function takes 2 arguments: `boardSize` which we'll use in our for...loop to traverse all cells in our grid and `cellsPlayed` which we'll use to identify which player made that move when traversing through each of our cells.

```typescript
   // 1. Check rows
  let doesRowMatch = true;
  let rowCheck: PlayerTypes | undefined
  for (let i = 0; i < boardSize; i++) {
    rowCheck = cellsPlayed.get(`${i}${0}`) || "-";
    for (let j = 1; j < boardSize; j++) {
      if (rowCheck !== cellsPlayed.get(`${i}${j}`)) {
        doesRowMatch = false;
        break;
      }
    }
    if (doesRowMatch) {
      // exit, since we already found a winner.
      break;
    }
  }
```
Here, we loop through every single row in our array to check if there's a row where all cells match. The assumed state is that all cells will pass. If we run into a row with cells that do not pass, we break out of the inner loop since this entire row is already compromised. The if statement after the inner loop checks whether we did find a match. If we did, we do not need to traverse any other cells and instead we break out of loop. To better picture this, consider that the array we're working with is a matrix of `boardSizeXboardSize` and is a square. Here's an illustriong with indices: [matrix](https://drive.google.com/file/d/1xrFmtXTz39yZxjfK1PaNv1EiJNQ8ioew/view?usp=sharing).
The time complexity here is O(n*m) since we have a nested for...loop.

```typescript
  let doesColumnMatch = true;
  let columnCheck: playerTypes | undefined;
  if (!doesRowMatch) {
    for (let i = 0; i < boardSize; i++) {
      columnCheck = cellsPlayed.get(`${0}${i}`) || "-";
      for (let j = 1; j < boardSize; j++) {
        if (columnCheck !== cellsPlayed.get(`${j}${i}`)) {
          doesColumnMatch = false;
          break;
        }
      }
      if (doesColumnMatch) {
        // exit, since we already found a winner.
        break;
      }
    }
  }
```
Here, we repeat the same logic for rows for columns. The only difference here is that we have inverted the indices when coming up with our keys. i.e. for the rows, originally it would have been `[i][j]` but for this we instead do `[j][i]`. Again, this is so much easier to picture if you consider our 2D array as a grid/matrix.
The time complexity here is O(n*m) since we have a nested for...loop.

```typescript
 // 3. Check left diagonal
  let doesLeftDiagonalMatch = true;
  // Duplicate indices (i.e. 00, 11,22 ...) and you'll get your left diagonal.
  const leftDiagonalCheck = cellsPlayed.get(`${0}${0}`) || "-";
  if (!doesColumnMatch) {
    for (let i = 1; i < boardSize; i++) {
      let nextLeftDiagonalCheck = cellsPlayed.get(`${i}${i}`);
      if (leftDiagonalCheck !== nextLeftDiagonalCheck) {
        doesLeftDiagonalMatch = false;
        break;
      }
    }
  }
```
For left diagonals, as long as the matrix/grid is a square, the indices will always behave such that `i` and `j` will always be the same. As such, we need only a single loop and we infer the value of `j` from `i`.
The time complexity here is O(n) since we have a single for...loop.

```typescript
  // 4. Check right diagonal.
  let doesRightDiagonalMatch = true;
  const rightDiagonalCheck = cellsPlayed.get(`${0}${boardSize - 1}`) || "-";
  if (!doesLeftDiagonalMatch) {
    for (let i = 1; i < boardSize; i++) {
      let nextRightDiagonalCheck = cellsPlayed.get(`${i}${boardSize - 1 - i}`);
      if (rightDiagonalCheck !== nextRightDiagonalCheck) {
        doesRightDiagonalMatch = false;
        break;
      }
    }
  }
```
For right diagonals, the behaviour is such that when value of `i` increase by 1, the value of `j` decrease by 1. As such, we also need only one loop here and we subtract the value of `i` from the total - 1 (`boardSize - 1`) to get indices of `j`.
The time complexity here is O(n) since we have a single for...loop.

```typescript
let currentWinner: playerTypes | undefined;
  if (doesRowMatch) {
    currentWinner = rowCheck;
  } else if (doesColumnMatch) {
    currentWinner = columnCheck;
  } else if (doesRightDiagonalMatch) {
    currentWinner = rightDiagonalCheck;
  } else if (doesLeftDiagonalMatch) {
    currentWinner = leftDiagonalCheck;
  }
  if (currentWinner) {
    alert(`Game over. ${toTitleCase(currentWinner)} has won! Click on the reset button to start over.`);
    return true;
  }
```
Lastly, if we did find any columns,rows or diagonals matching, we print out the winner. This function returns a boolean to indicate whether a winner was found or not. This return value is used when no more moves are left and there was no winner.

In conclusion, due to the performance implications of nested for loops (when traversing cells in a 2D array), we have used maps and sets to store the values so that we get sublinear time complexity (O(log n)) or linear time (O(n)) since this is based on javascript but ideally a map and set lookup should be constant time (O(1)).

That's all for the tic-tac-toe game. Feel free to reach out if there's anything that's unclear, you have improvements or you'd just like to chat!


Tic-Tac-Toe: Exploring a TypeScript Code Example

If you are a django developer, you can find a really comprehensive tutorial here: [django on Docker](https://lewiskori.com/post/deploying-a-python-django-application-using-docker)

## Now Shall We?

Docker, the new sheriff in town whose new gold boots is getting everyone excited. Why? because Docker is development bliss since it ensures consistency on both your development and production cycles therefore standardizing your environment

I assume you're as excited about docker as the rest of us and hence explains why you have stumbled upon this post. I'll be honest, docker at first, without any prior knowledge is a nightmare, as any other new concept without prior knowledge, but once you do get the hang of it, it'll save you tonnes of time you'll have otherwise invested during deployment.

This post is intended mostly for beginners who probably have little or no knowledge about docker. The motivation behind this post lies on the fact that I wasn't able to find any comprehensive Laravel Docker tutorials online. I was only able in the end to accomplish this task through continuous reading of multiple blog posts and combining all this into a massive comprehensive series of steps that I'll attempt to document in this post.

## Now Onto The Good Stuff..

Before I start, I assume that you already have your Laravel Application ready with you. If you don't, you can jump on to the [Laravel Documentation Page](https://laravel.com/docs/5.7/) and build yourself an app, then head back here and continue reading.

I also assume that you already have Docker installed on your machine. In case you don't, you have the following options:

1. Windows 10 Pro Users: [Docker Desktop](https://docs.docker.com/docker-for-windows/install/)
2. Windows 10 Version that's below Windows 10 Pro Users: [Docker Toolbox](https://docs.docker.com/toolbox/toolbox_install_windows/) .This is because of Docker Desktop system requirements. Docker Toolbox leverages on the functionalities of VirtualBox.
3. Linux Users: [Docker CentOS](https://docs.docker.com/install/linux/docker-ce/centos/) . You can choose your Linux distribution (if its not CentOS) from the side menu on the screen that comes up and follow those distribution specific instructions.

## First Step: Creating your 'docker-compose' file

What is a docker-compose file? This is a file that defines all your multiple docker containers and all these containers can be spawned up by running a relatively simple command like:

```
docker-compose -f docker-compose.prod.yml up --build
```

Here, we'll be setting up a development environment (Which can also be used in your production environment, with a few minor changes that I'll document in my next post :-))

Create a file in your root directory and name it: `docker-compose.yml`

You'll be defining the containers in the next steps in this file.

In our docker-compose file, we define three containers: mysql, nginx and our laravel app.

### So, for starters, our laravel app container will be defined as follows:

```
version: '2'

services:

  #  The Application
  app:
    container_name: laravel_app
    build:
      context: ./
      dockerfile: development/app.dockerfile
    volumes:
      - ./storage:/var/www/storage
    env_file: '.env.prod'
    environment:
      - "DB_HOST=database"
      - "REDIS_HOST=cache"
```



**Overview of the Above Code:**

1. **version** - Feel free to change this to your choosing
2. **container_name** - You'll use this name to refer to your container. i.e. if you'd want to close your container, you'll use this name to refer to it specifically. Also feel free to change it to your choosing.
3. **build** - Used to build an image from a Dockerfile. Has the following additional options:
  - _Context_ - Docker uses this context (basically, where your laravel files reside) to reference any files within it. In this case, the ./ refers to the root laravel folder assuming that the docker-compose file is stored in your laravel root folder.
  - _dockerfile_: docker images are built from Dockerfiles, which often contain additional commands that should be run inside the container. In this case, the dockerfile we use to build our **app** container. Also note that we have used `development/app.dockerfile` .This means that our docker file is located in a 'development' folder on the root of our laravel app.

4. **volumes** - Volumes are used by docker containers to share files between the host machine and the docker container that is running. The left hand side of the full colon represents our host machine and the right hand side represents our docker container. In this case, we're sharing all data in the storage folder on our laravel app with the docker container mounted at _/var/www/storage_
5. **env_file** - This defines our laravel's .env file, in our case env.prod that we'll use to input docker container specific environment variables as we'll see later on this post.
6. **environment** - This defines the environment variables that will be set on our docker machine. In this case, if we can execute a bash command inside our linux container and reference the environment variables we define here, i.e. `echo $DB_HOST` will print out: **database**


### Our NGINX Container will be defined as follows:

```
# The Web Server
  web:
    container_name: nginx_server
    build:
      context: ./
      dockerfile: development/web.dockerfile
    volumes:
      - ./storage/logs/:/var/log/nginx
    ports:
      - 8990:80
```

**Overview of the Above Code:**
1. **container_name** - Again, the name of your container, which you can choose to change.
2. **build** - Definition same as above. Here you can see that we define this container's dockerfile as web.dockerfile.
3. **volumes** - Definition same as above. Here we share our laravel's logs folder with nginx's logs folder.
4. **ports** - Here, we define the port in the host machine that our docker container will be listening on and the port on the virtual network created by docker during container deployment. This can be easily visualised by understanding that the left side of the colon defines the host machines, therefore ports on the host machine and the right side of the colon the docker container, therefore the ports on the docker container.

### Our MySQL Container will be defined as follows:

```
# The Database
  database:
    container_name: mysql_database
    image: mysql:5.7
    volumes:
      - dbdata:/var/lib/mysql
    environment:
      - "MYSQL_DATABASE=Baly"
      - "MYSQL_USER=phpmyadmin"
      - "MYSQL_PASSWORD=phpmyadmin"
      - "MYSQL_ROOT_PASSWORD=finallyJDBC2017."
    ports:
      - 8991:3306
```

**Overview of the Above Code:**
1. **container_name** - Refer to above.
2. **image** - In this case, we haven't defined a dockerfile to base our container build on, but rather an image. Our docker container will therefore be built from the image we've defined in this case `mysql5:7` image. You can switch this mysql version with the version you're developing with. Remember that, with reference to your laravel application, the newest versions of mysql may not work with your laravel app. This is because the newest versions MySQL use a different technique of authentication that may not be supported by either mysql or pdo php extensions. Therefore, beware when invoking `mysql:latest` instead of  `mysql:5.7`.
3. **volumes** - Still the same concept, except that now we've defined `dbdata` from our host machine that will map to `/var/lib/mysql` on the docker container.
4. **environment** - Same concept as defined above, except that in this case, our mysql database will be initialized with the variables we have set. Therefore, our container after build, will automatically have a database named `database` , a user named `secret` identified by the password `secret` and a root password of `secret_root`. You can feel free to change these as you please. We define these settings in our `env.prod` file so as not to collide our current `.env` file settings with our container file settings.
5. **ports** - same as above, except that our mysql container will be listening on port 8991 on the host machine and 3306 (mysql's default port) on the container's network.

### Defining your named volumes

Copy paste the following into your docker-compose.yml file:
```
volumes:
  dbdata:

```

Ensure that you need to preserve the indenting in your docker-compose.yml file to ensure that docker-compose reads it correctly. In the end, your docker-compose file should look as follows:
```
version: '2'

services:

  #  The Application
  app:
    container_name: laravel_app
    build:
      context: ./
      dockerfile: development/app.dockerfile
    volumes:
      - ./storage:/var/www/storage
    env_file: '.env.prod'
    environment:
      - "DB_HOST=database"
      - "REDIS_HOST=cache"

  # The Web Server
  web:
    container_name: nginx_server
    build:
      context: ./
      dockerfile: development/web.dockerfile
    volumes:
      - ./storage/logs/:/var/log/nginx
    ports:
      - 8990:80

  # The Database
  database:
    container_name: mysql_database
    image: mysql:5.7
    volumes:
      - dbdata:/var/lib/mysql
    environment:
      - "MYSQL_DATABASE=Baly"
      - "MYSQL_USER=phpmyadmin"
      - "MYSQL_PASSWORD=phpmyadmin"
      - "MYSQL_ROOT_PASSWORD=finallyJDBC2017."
    ports:
      - 8991:3306

    # redis
  cache:
    image: redis:3.0-alpine

volumes:
  dbdata:

```

## Second Step: Defining our Dockerfiles.

In this step, we define the dockerfiles for the containers we just defined in our docker-compose file. These dockerfiles will represent a series of commands that we'll want to run inside our docker containers.

### Defining our 'app' dockerfile (laravel_app)

Create a folder in your laravel app's root directory and name it `development`. Inside the folder you just created, create a file and name it `app.dockerfile` (yes, without any extensions). Open this file and copy paste the following code into it:

```
FROM php:7.2-fpm

COPY composer.lock composer.json /var/www/

COPY database /var/www/database

WORKDIR /var/www

RUN apt-get update && apt-get -y install git && apt-get -y install zip

RUN php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');" \
    && php -r "if (hash_file('SHA384', 'composer-setup.php') === 'a5c698ffe4b8e849a443b120cd5ba38043260d5c4023dbf93e1558871f1f07f58274fc6f4c93bcfd858c6bd0775cd8d1') { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;" \
    && php composer-setup.php \
    && php -r "unlink('composer-setup.php');" \
    && php composer.phar install --no-dev --no-scripts \
    && rm composer.phar

COPY . /var/www

RUN chown -R www-data:www-data \
        /var/www/storage \
        /var/www/bootstrap/cache

RUN  apt-get install -y libmcrypt-dev \
        libmagickwand-dev --no-install-recommends \
        && pecl install mcrypt-1.0.2 \
        && docker-php-ext-install pdo_mysql \
        && docker-php-ext-enable mcrypt

RUN mv .env.prod .env

RUN php artisan optimize
``` 

**Overview of the Above Code**
1. **From php:7.2-fpm** - This means will be building our container from an image, `php:7.2-fpm`. Also, you can change this version to meet your development environment needs.

2. **COPY** - In the first copy command, we copy our `composer.lock` and `composer.json` from our root folder (in our host machine) to `/var/www/` in the docker container. In the second copy command, we copy our `database` folder in the host machine to `/var/www/database` folder in the docker container. This is because, one, we'll want to make sure that the dependencies we use in our development environment (in composer.json) will be reflected inside the container when we download dependencies  and two, that we can access our migrate files inside the docker container in cases we may need to run `migrate` command.

3. **WORKDIR** - We set the working directory to `/var/www` which means we don't have to `cd` to this folder (move to this folder) in cases we'll need to run bash commands.

4. **RUN** - Here, we install all the dependencies that will be needed by laravel, including composer and the dependencies needed by composer. Please note the `if(hash_file('SHA384'...` line. The hash value defined there will change with every update, and therefore if your installer fails with the message: **installer corrupt**, consider getting the correct hash value from: [Get Hash Value](https://getcomposer.org/download/).

5. **COPY . /var/www** - At this point we copy all our folder contents into `/var/www` folder in the docker container.

6. **RUN** - In the final run commands, we clear our application cache and other cache and install the mysql driver that laravel uses to make connections to the database. Afterwards, we rename our `.env.prod` file to `.env` since this file will contain the correct environment variables specific to the docker container environment and therefore should be used by laravel. We run `php artisan optimize` to remove the cached version of the `.env` file.

Please note that it is unnecessary to copy everything from our root folder (like vendor folder) and docker provides a `.dockerignore` file which works pretty much like a `.gitignore` file. Our dockeringore file will look as follows:

```
.git
.idea
.env
node_modules
vendor
storage/framework/cache/**
storage/framework/sessions/**
storage/framework/views/**
development
```

Save this file in the same folder as your app.dockerfile (development folder).

For your `.env.prod` file, copy paste your `.env` file and rename it to `.env.prod`. In the database settings, change the `DB_HOST` to match the name of your **mysql** container, and the password to match what you defined in your `docker-compose.yml` file. If you followed all my steps without changing a thing, then your .env.prod file should resemble the following:

```
DB_CONNECTION=mysql
DB_HOST=mysql_database
DB_PORT=3306
DB_DATABASE=Baly
DB_USERNAME=phpmyadmin
DB_PASSWORD=phpmyadmin
```

### Defining our 'web' dockerfile

In the same folder you just created (the development folder) create a `web.dockerfile`. Copy paste the following to the dockerfile:

```
FROM nginx:1.10-alpine

ADD development/vhost.conf /etc/nginx/conf.d/default.conf

COPY public /var/www/public

```

**Overview of the Above Code**

We build our dockerfile from the image: `nginx:1.10-alpine`. We then replace nginx's `default.conf` file with the `vhost.conf` we'll create in a sec.

We also copy our laravel app's public directory to the public directory of nginx, that will server all our public assets.

**Create a vhost.conf file in this same directory (development) and copy paste this into it:**

```
server {
    listen 80;
    index index.php index.html;
    root /var/www/public;
    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;
    location / {
        try_files $uri /index.php?$args;
    }

    location ~ \.php$ {
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_pass app:9000;
        fastcgi_index index.php;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param PATH_INFO $fastcgi_path_info;
    }
}

```
our `php-fpm` container will be listening on port 9000 and hence `app:9000`


## Almost there...

So, to counter-check, you need to already have the following files:
1. **Root folder** - `docker-compose.yml` and `.env.prod`
2. **development folder**:
  - `.dockerignore`
  - `app.dockerfile`
  - `web.dockerfile`
  - `vhost.conf`

### If so, then you're almost done, but first, some prerequisites:

If you are using Docker Toolbox on Windows and your laravel app folder is in a folder other than `C:/users` you will have trouble sharing volumes between your host machine and your docker containers. This is because any other folder that's not `C:/users` is not mounted by virtual box when your docker machine starts. Therefore, to fix this, first stop your running docker machine by running:

```
docker-machine stop
```

Then open virtualbox, right-click the machine named **default**  and click on **settings** . Navigate to **Shared Folders** click on it, and add a new folder that defines the location of your laravel app folder. Remember to check against **Auto mount**. Afterwards, start your docker machine by running:

```
docker-machine start default
```


## Drum Rolls...

Assuming you have done everything correctly, go ahead and run the following command:
```
docker-compose up -d --build database && docker-compose up -d --build app && docker-compose up -d --build web 

```

finally, enter you laravel_app's docker container by executing:

```
docker exec -it laravel_app bash
```

And execute the following laravel commands:

```
php artisan key:generate

php artisan config:cache

php artisan route:cache

```

Make sure that you are running this command inside the root folder of your laravel app. This command builds your container images and finally starts them. If everything goes according to plan, you should be able to access your laravel app running inside your container at:


```
0.0.0.0:8990
```
Replace `8990` with the port you defined in your docker-compose.yml file if you used a different port.

Also, please note that for users using Docker Toolbox, docker creates a virtual network and assigns an IP address to it. You can find this IP address by searching for `docker quickstart terminal` and running it. The IP address assigned will be displayed in the terminal that pops up and you'll be able to access your laravel app by going to:

```
your-docker-machine-ip:8990
```

And there you have it folks! You have successfully deployed your laravel app on docker! Stay tuned for my next post where I'll be describing on how to deploy your Laravel app on a production environment.


Deploying Your Laravel App on Docker, With NGINX and MySQL

An Introduction to Spring Boot: MySQL, JPA and Hibernate.

An Introduction to Spring Boot: MySQL, JPA and Hibernate

There has been a lot of hype around this mysterious new complex technology of storing and exchanging money that is decentralized, distributed and public. **Well, Spoiler-alert: It's not as complex as most of us think.**

To be able to understand this technology, we'll be looking at Bitcoin as our case study, The most popularly used cryptocurrency that implements Blockchain technology under the hood.

This will be a 3 part series that will comprise of:

1. Blockchain Technology: A case study of Bitcoin
2. Implementing Blockchain Technology in Code
3. Understanding the Math Behind Bitcoin

Buckle up and let's get into it.

## So What is Blockchain Technology?
At its most simplified existance, blockchain basically defines a chain of blocks whose blocks store digital data and are interlinked with each other in a unique way. Let me explain further:

**1. Blocks** - They store Transactional data (i.e the way e-commerce sites store your transaction data like the date, amount and your name). So a single block can be defined as a group of transactions. These blocks also store unique information about themselves (known as a hash - a unique string of numbers and letters) that are used to identify the blocks in the blockchain.
**2. Blockchain** - These are a group of blocks that are interconnected with each other, pretty much like a database containing a list of blocks as its entries.

Now lets get deeper.

## How does Bitcoin implement Blockchain Technology?

For starters, as long as you ended up here, you know that bitcoin is the most popular cryptocurrency currently in use - Well, as of 17th October 2019 - Technology is very dynamic and so this might not be the case a few years later. Either way, most new cryptocurrencies out there are a fork of Bitcoin technology with a few tweaks here and there to try to overcome the limitations of Bitcoin. So at this point, I'd like to assume that it's safe for me to call Bitcoin the father of other cryptocurrencies.

So how does Bitcoin do it? Well, all this boils down to Cryptography (The act of securing communication between two points), Math, Technology and a [pseudonymous](https://www.google.com/search?q=pseudonymous+meaning&oq=pseudonymous+meaning&aqs=chrome.0.69i59j0l5.1175j0j4&sourceid=chrome&ie=UTF-8) Japanese person named Satoshi Nakamoto.

Let's consider bitcoin as a decentralized, distributed, public ledger. Decentralized in the form that no central bank controls controls it, Distributed such that every peer on the network contains a copy of the ledger and public also such that every peer on the network has access to this distributed ledger.

Now, let's also consider two popular individuals: Alice and Bob, who are trying to transact using bitcoin. So both these individuals start off which getting a Bitcoin Wallet - A program responsible for managing bitcoins which generates a private and public key pair where the private key is responsible for authorising transactions from your bitcoin wallet and a public key that is used as your public address when sending and receiving bitcoins. The public Key is generated from the private Key.

(You can read more about [What is a bitcoin wallet here](https://www.youtube.com/watch?v=A1Pl5hYHXiI) and [the differences between a bitcoin client and a wallet here] (https://bitcoin.stackexchange.com/questions/20487/whats-the-difference-between-a-bitcoin-client-and-wallet))

**So What is a private key?** A private key is a uniquely generated string of letters and numbers that will be used to authorise your bitcoin transactions (Basically, like a pin to your ATM Card).
**What is a public key?** A public key is also a uniquely generated string of letters and numbers - generated from the public key - that when hashed, will be used as your address when sending or receiving money. So this means for Alice to send Bob money, Alice will need to know Bob's public key.
**What is Hashing** - Well hashing is the process of converting a certain string of characters to a unique string of letters and numbers of a certain length through a hashing function. As an example, given a String `john` passed through a SHA256 hashing function, you get `96D9632F363564CC3032521409CF22A852F2032EEC099ED5967C0D000CEC607A` which has a total of 64 characters which is equivalent to 256 bits. If you are not a computer Guru - well, most of us aren't - I'll try to explain the math briefly. In computers, every single character in Hexadecimal notation has a total of 4 bits, ranging from 0-9 and A-F. Bits are basically zeros and ones which computers use to represent data. So in this case, the hash generated is a length of 64 characters and since every character is 4 bits, the total becomes 256 bits. This length of 256 bits produced will always be the same regardless of what you feed into the SHA256 hashing function. So even a String longer than `John`, say `Micheal` would still yield a 256 bit long String, except that it would be quite different from what was gotten when using `John`

So Alice and Bob have successfully acquired their Bitcoin wallets. Bob then decides to gamble with his rent money by betting on racing horses. Well long story short, the Horse Bob bet on wasn't actually a horse so Bob looses all his rent money. So Bob is on the verge of being kicked out of his home and at the same time is considering betting on a football match that comes on 4 hours later and therefore reaches out to Alice for financial help. Alice, having being brought up in a sharing family, decides to send Bob 2 bitcoins. So to do this, Alice needs Bob's Bitcoin address (which as we had discussed earlier, is Bob's hashed public key). So Bob sends Alice this address and Alice decides to send 2 Bitcoins to Bob.

To be able to do this, another important concept about Bitcoin should be understood, and this is that bitcoin does not store people's balances. Bitcoin rather stores a list of all transactions that have taken place since it's birth. Therefore, to prevent Alice from sending money that she doesn't have, the Bitcoin  client will look at previous transactions that Alice was involved in (Referred to as inputs in bitcoin terminology) and check whether Alice received a sum-total that is either equal to or greater than the amount Alice wants to send, in this case 2 Bitcoins and also whether these transactions were already involved in any other transaction. If the inputs pass these parameters, then it is verified that Alice has at least 2 bitcoins in her account and can be able to send them. Also, it is important to note that not always will the inputs be exactly equivalent to the amount Alice wishes to send and therefore the surplus amount will be sent back to the Sender. So if Alice's previous transactions add up to a total of 3 bitcoins, then the extra bitcoin will be sent back to Alice.

Alice is the heir to a massive estate and therefore very wealthy, and Alice recalls that she had recently converted a couple millions to bitcoins and had this money sent to her address by the bank she was using. So Alice is quite confident that she has the capability of sending Bob 2 Bitcoins, so Alice authorises the transaction.

To be able to authorise and validate transactions, we need to understand that for starters, the bitcoin network is an interconnection of multiple nodes all containing the public list of transactions. These transactions are grouped together in blocks and this blocks are inter-linked together to form a block-chain. Each block has a unique hash that identifies it (generated from the transactions in that block together with the hash value of the previous block) and every next block in the block-chain stores this hash value of the previous block. This way, Blocks cannot be swapped with other blocks within the block chain since new blocks will always have a different hash value. At the same time, since the block's hash value is generated from the list of transactions, changing a single transaction in a block will change the hash value of that block and subsequently all the other hashes of following blocks. This is therefore an important security feature of blockchain technology that we'll look at later.

So Alice inputs Bob's bitcoin address and presses the send button on her wallet. This generates a transaction that is added to a list of unverified transactions on the bitcoin network - So this transaction hasn't been added to any block, and therefore not on the block-chain, and therefore not on the public list of transactions. For this transaction to be considered valid, the nodes on the network will first check whether the inputs (As mentioned earlier, previous transactions) have been used in any other prior transactions. At the same time, these nodes check on whether the send to address is valid. So in general, the Bitcoin network performs the following tasks before considering a transaction/group of transactions are valid:

### Steps run in the Bitcoin network:
**1) New transactions are broadcast to all nodes** - Alice's transaction is broadcast together with other transactions currently taking place, to all other nodes in the network. Nodes here are Bitcoin clients running connected to the network. Bitcoin clients are programs that you install on your machine that perform these tasks.
**2) Each node collects new transactions into a block**
**3) Each node works on finding a difficult proof-of-work for its block** - Here, proof of work defines a mathematical problem that a node needs to solve in order to mark a block of transactions as valid. I'll talk more about this in the next section
**4) When a node finds a proof-of-work, it broadcasts the block to all nodes**
**5) Nodes accept the block only if all transactions in it are valid and not already spent**
**6) Nodes express their acceptance of the block by working on creating the next block in the
chain, using the hash of the accepted block as the previous hash**

### So what is this proof-of-work jargon?

Well, the answer to this question my also be the answer to other questions you might have: like where does bitcoin come from - can also rather be phrased as where did the first bitcoin come from - or does Alice have a thing for Bob, or whether Bob has a gambling problem and should sick professional help. Either way, hold on to the railings cause this Titanic is about to hit an iceberg.

So as we mentioned earlier, every pending transactions is held in a virtual pool that can be identified as a pool of unidentified transactions. The nodes in the bitcoin network are responsible for verifying these transactions and putting them inside a block. These nodes also attempt to solve a mathematical problem through which they use the correct answer to this problem to mark a block as valid. In the end, the node that does this first, gets rewarded for the work in form of bitcoins. As we also mentioned earlier, bitcoin uses a hashing algorithm to generate addresses. Bitcoin then uses this same technique to generate a mathematical problem, say generate a certain hash value whose first 32 bits are zero. Since such problems can only be solved through brute-force (continuously guessing using random values until you achieve the desired result) this makes the probability of solving such guesses at 2^32 since a single bit can only have two states, 0 and 1, and this value equates to above 4 Billion guesses! So there is a single solution every 4 billion guesses. A node on the bitcoin network approximately solves a mathematical problem every 10 minutes so bitcoin fine-tunes the mathematical problem such that this aspect is maintained. So this entire phenomena of solving a mathematical problem so as to validate a single block is what is referred to as **Bitcoin Mining**.

After all parameters of Alice's transaction check out, the transactions is considered valid and added to a block that has been validated and Bob is free to either pay his rent, or make another betting blunder. At the same time, a certain bitcoin amount is sent to the minor - pretty much like a transaction cost - As a thank you to the miner for devoting their computer's precious time and power to solving blocks. Because the transaction has been sent to Bob's public address, only Bob has his private key and therefore he's the only individual who can spend the bitcoins associated with his public address. Therefore, whoever has the private key - Stored locally by the bitcoin wallet up as a wallet.dat file - controls the bitcoin, So you should consider keeping your private key as private as possible.

**This is how bitcoin greases it's continually moving gears in the bitcoin network. Wasn't so hard after all was it?**

If you're a sceptical type of individual, you may have questions about whether or not this network controlled by everyone is secure on not, so in this next section, we're going to look at Security issues in bitcoin and how Bitcoin overcomes them.

### Security issues - Also contains a somewhat more technical explanation of 'proof-of-work'
### 1. Transaction Order:
Every new Bitcoin transactions is broadcast-ed across the network to all other nodes. In this case, Bitcoin is decentralised such that every Bitcoin client contains a copy of all the transactions made in Bitcoins. Therefore, you cannot always accurately state that a transaction reaches all nodes in the bitcoin network at the same time. Consider an example: Alice wants to buy a car from Bob. Alice makes the transaction and sends Bob 5 Bitcoins for the car and waits for Bob to ship the car. Afterwards, Alice then uses the same inputs she used to send Bob 5 bitcoins and send herself back the money. Due to different network propagation times, some nodes will receive Alice's second transaction of sending herself back the 5 bitcoins before they receive Alice's transaction of sending Bob 5 bitcoins. These nodes would then consider the second transaction they received as invalid since they would be re-using inputs.

#### How does bitcoin overcome this?
Bitcoin arranges all transactions in a block (a limit of around 2400 transactions per block). These blocks are connected together in a chain, where every block references the previous block. You can therefore traverse these blocks in order, to the first transaction ever made in bitcoin. This therefore solves the problem of order, **but in what way are these transactions added to each block and in the correct order?** In this case, nodes take the **'un-ordered'** or **'unconfirmed'** transactions and puts them in a new block and broadcasts this block to the rest of the network. The un-ordered or unconfirmed transactions are transactions recently completed but have not been included into any block yet. But this also creates a problem in that multiple nodes could create their own Blocks containing the same transactions and broadcast them to the rest of the network. **So how does the network know which block to choose?** Bitcoin dictates that each valid block must contain an answer to a specific mathematical problem. Bitcoin therefore uses cryptographic hash functions, which are basically functions that when given a message as input, gives a random string of letters and numbers of a specific length (called the **message digest**) that is unique to this input as the output. This output will always be the same given that the input does not change. Changing even a minor part of the input will cause the output to change completely. These functions are referred to as **one-way functions** since you cannot use the output to inversely try to retrieve the input. In this case therefore, the only way of discovering the input is to run multiple guesses at a time until an output that matches the given output is produced. Bitcoin uses an algorithm: SHA256 that generates an output of the length of 256 bits. The mathematical problem that the nodes now have to solve is generate a certain hash value whose first n bits are all zeros (n can be any numerical value). To put this more into perspective, the total number of guesses that can be made is 2^n guesses, depending on the value of n. if n was 32, the total number of guesses would be 2^32 which is above 4 billion guesses, as we had calculated earlier. Therefore, bitcoin network fine-tunes the value of n such that the correct input will be produced around every 10 minutes. As we stated earlier, This phenomena is referred to as **Bitcoin mining**

The mathematical probability of this problem makes it very rare for more than one node to arrive at the correct input at the same time (i.e if n was 32, the probability would be one in over 4 billion guesses) but in cases nodes do overcome this probability and solve the problem at the same time, both blocks generated by these nodes will be considered valid and will be added to the block chain and the block's hash be used in the next block. This therefore raises another problem where there will be duplicate blocks in the chain so what the network does, is accept the longest block-chain as the proof-of-work. It is highly improbable that other nodes will again end up at a solution at the same time for the next block (due to how the mathematical problem is constantly fine-tuned) and therefore, there will always be a longer block chain and this chain will be considered as the valid chain by other nodes and therefore duplicate chains will be discarded. The reasoning behind this is that as long as honest nodes occupy the majority of the bitcoin network, the longest chain will be attributed to majority nodes working towards solving the mathematical problem (guessing correct values of the n number of zeros) therefore winning against all other competing chains in the network. Therefore, the longest chain will certainly always be produced by the majority number of nodes since the more the nodes trying to solve the problem, the higher the probability that the problem will be solved faster and therefore the block chain will grow faster. i.e., in the case of 4 billion guesses, a group of 1 million nodes will achieve the solution faster than a group of 1 thousand nodes, which makes it even harder for a single attacker with one node to win the race against the entire network.

It is therefore advisable to wait for more blocks to be added to the block chain before being sure that a transaction is actually valid since it becomes significantly harder to generate a longer block chain the further back you go in the chain. Consider this, our chain has 4 blocks already and the majority nodes are adding an extra valid block every 2 minutes, so for an attacker to be able to control this network, they need to have enough computing power to generate newer blocks faster than the network itself. So if the block chain grows to 8 blocks and the attacker has their copy of 4 blocks, they have to create the subsequent missing 4 blocks before they get to 8 blocks. Since the other nodes in the network are still actively creating blocks, this makes it very tough for the attacker to catch-up to the speed of the network the further back in the block-chain the attacker is.


Whoa! That's a whole lotta information to take in but I'd like to assume that deep down, you feel that it was worth it.
Thanks for reading and if you have any questions or clarifications, don't be afraid to leave a comment behind:-)
Also, stay tuned for the next post in this series: **Implementing Blockchain Technology in Code**.
You can also choose to read more on Bitcoin on the original paper posted [here](https://bitcoin.org/bitcoin.pdf)

#### au revoir:-)


Blockchain Technology: A case study of Bitcoin