One of the many things you do in a typical app is to enable users to store files. While some find it easier to store files in database (huh?), the most popular option is to store files and reference them in the database record.

Firestore provides easy access to a service known as Firebase Storage to store files. Let’s see how.

Create Vue View

We get started with a simple Vue app (which is using Vuetify, but that does not matter)..

<template>
  <v-container>
    <v-row>
      <v-col cols="12" class="mt-12 pt-md-6" align="center">
        <span class="title font-weight-black">Image Handler</span>
      </v-col>
      <v-col cols="12" class="mt-12" align="center">
        <v-file-input
          v-model="myFile"
          outlined
          :rules="[rules.maxsize]"
          accept="image/png, image/jpeg"
          placeholder="Click to upload file"
          @change="fileInput"
          :disabled="processing"
        >
          <template v-slot:append-outer>
            <v-progress-circular
              v-if="processing"
              color="grey"
              indeterminate
              small
            />
          </template>
        </v-file-input>
      </v-col>
      <v-col cols="12" align="center">
        <v-img :src="fileUrl" contain v-if="fileUrl" max-height="500"></v-img>
      </v-col>
    </v-row>
  </v-container>
</template>

<script>
  import { mapState, mapMutations } from "vuex";
  import { FirebaseFunctions, FirebaseStorage } from "@/services/firebase.js";

  export default {
    data() {
      return {
        myFile: null,
        processing: false,
        fileUrl: null,
      };
    },
  };
</script>

Add a Service to invoke Firebase

Install Firebase in your vue app.

npm i firebase

Add a service that can be consumed by any Vue components to upload files - src/services/firebase.js.

import firebase from "firebase/app";
import "firebase/storage";
// get config from Firebase console
const firebaseConfig = {
  apiKey: "the_api_key",
  authDomain: "myapp.firebaseapp.com",
  databaseURL: "https://myapp.firebaseio.com",
  projectId: "myapp",
  storageBucket: "myapp.appspot.com",
  messagingSenderId: "12345",
  appId: "1:123512:web:1abcd23",
  measurementId: "G-123COOL",
};

firebase.apps && !firebase.apps.length
  ? firebase.initializeApp(firebaseConfig)
  : "";

export const FirebaseStorage = firebase.storage();
export default firebase;

We add the above code as a service instead of using them locally in components (or views) since we may need the same code in multiple places.

A few key points to note -

• Get configuration from your Firebase console. This forms the entirety of firebaseConfig object
• We export FirebaseStorage (alongside firebase) and this a ready-to-use service to access the Firebase storage

In the next section you will see how we use this service.

Add Method to Upload File

Finally, add the methods to handle image uploads in the view.

<script>
  import { FirebaseStorage } from "@/services/firebase.js";

  export default {
    data() {
      return {
        myFile: null,
        processing: false,
        fileURL: null,
      };
    },
    methods: {
      async fileInput(file) {
        try {
          if (file && file.name) {
            this.processing = true;

            const fr = new FileReader();
            fr.readAsDataURL(file);
            fr.addEventListener("load", () => {
              // this is to load image on the UI
              // .. not related to file upload :)
              this.fileURL = fr.result;
            });
            const imgData = new FormData();
            imgData.append("image", this.myFile);
            const filePath = `mypath/${Date.now()}-${file.name}`;
            const metadata = { contentType: this.myFile.type };

            await FirebaseStorage.ref()
              .child(filePath)
              .put(this.myFile, metadata);
            console.log("filePath: ", filePath);
          }
        } catch (e) {
          console.error(e);
        } finally {
          this.processing = false;
        }
      },
    },
  };
</script>

We are doing a few simple things here -

• We read the file using const fr = new FileReader(); and display the file on the UI. This is just a “nice to have” feature to provide feedback to user when she selects the file
• The selected file gets appended to form data - const imgData = new FormData()
• Upload the file to the path const filePath = ``mypath/${Date.now()}-${file.name}``; on our Firebase storage using FirebaseStorage.ref()

That is it! You can now upload any file to Firestore (provided your security rules allow you to upload files).

The first thing that comes to your mind is to just use the browser feature to print any web page (or to save as PDF). Well, it gets the job done but along with your beautiful, print-worthy content will be the buttons, footer, and what not. So, you get to wonder “is it possible to get rid of things only for prints but not on the web page UI”? The answer is “yes”, of course - I am writing a whole post about it.

And, a whole post about a couple of simple statements in CSS that can make elements magically disappear.

@media print {
  .no-print {
    visibility: hidden;
  }
}

All this will do is to instruct browser to hide any element with class no-print - only while printing the page (that’s the @media). This method works independent of frameworks and you can use it to hide elements in Vue, React, et al.

For e.g. consider this page.

<div id="app">
  <v-app>
    <v-toolbar>
      Meet my Friends
      <v-spacer></v-spacer>
      <v-btn outlined class="mr-2">Exit</v-btn>
      <v-btn color="primary">Know More</v-btn>
    </v-toolbar>

    <v-container>
      <v-row>
        <v-col
          v-for="item in items"
          :key="item.id"
          class="mt-2 mb-2"
          color="grey lighten-3"
          flat
          cols="12"
          md="4"
        >
          <v-toolbar flat dense color="transparent" class="font-weight-bold">
            {{item.title}}
          </v-toolbar>
          <v-img :src="item.img" height="300px"></v-img>
        </v-col>
      </v-row>
    </v-container>
    <v-footer>Images are from Unsplash.</v-footer>
  </v-app>
</div>

The toolbar and footer tag along for the prints.

Let us change that. Include the simple CSS provided earlier and use the CSS class against all elements that do not need to be printed.

For e.g. we do this to hide the toolbar -

<v-toolbar class="no-print">
  Meet my Friends
  <v-spacer></v-spacer>
  <v-btn outlined class="mr-2">Exit</v-btn>
  <v-btn color="primary">Know More</v-btn>
</v-toolbar>

The end result is a cleaner print-out (or PDF) with no unecessary elements.

Simple and elegant.

Find the code on Codepen.

From get go, I liked Fiber framework.

Fiber is an ExpressJS inspired framework -

• easy for NodeJS developers to pick up
• fast (I mean really fast - see benchmarks)
• easy to develop
• flexible

Coupled with Go’s simplicity, light-weight nature, ease of async programming and deployment, Fiber is a formidable (albeit simple) tool to quickly develop web apps.

Also see: a small-scale developer’s view of NodeJS vs Golang.

In this post, let us see an example of developing simple CRUD APIs in Fiber v2. We will not be using an ORM or a database to keep things focused on Fiber and Golang.

Start with Go

First, download and install Go on your computer.

Ensure that your favourite editor (= VSCode) has support to code Go programs.

• Install Go extension
• Configure editor

If you are looking at this in 2020, you may also need to enable Gopls in VSCode to enable better module support and more modern features. Hit Ctrl + Shift + P > type Settings > Hit Enter. Type in @ext:golang.go gopls to see Go extension settings. Scroll down and enable Go: Use Language Server.

You are all set.

Project Structure

Create a folder on your computer and open that folder in VSCode.

Create a new file server.go. Code in the following -

package main

import "fmt"

func main() {
fmt.Println("hello world")
}

Open terminal in VSCode (Ctrl + ~) and run program.

go run server.go

You should see an output message if everything works fine.

Get started on Fiber

Enter below command in the VSCode terminal to install fiber.

go get github.com/gofiber/fiber/v2

We will now start coding the API in server.go.

package main

import (
	"github.com/gofiber/fiber/v2"
	"github.com/gofiber/fiber/v2/middleware/logger"
)


type Todo struct {
	ID        int    `json:"id"`
	Name      string `json:"name"`
	Completed bool   `json:"completed"`
}

var todos = []Todo{
	{ID: 1, Name: "abc", Completed: false},
	{ID: 2, Name: "def", Completed: true},
}

func main() {

	app := fiber.New()
	app.Use(logger.New())

	app.Get("/", func(c *fiber.Ctx) error {
		return c.SendString("Hello, World!")
    })

    app.Listen(":5000")
}

We have done a few simple things here -

• defined a basic structure for our todo using type Todo struct {}. We also provided the JSON equivalents of the fields
• created a simple array of type Todo - var todos = []Todo{{ID: 1, Name: "abc", Completed: false},}
• initiated fiber with app := fiber.New()
• enabled logging input requests with a single line app.Use(logger.New())
• enabled a simple response at root using app.Get("/", ...)

Run the program again using go run server.go.

Use a REST client (like Insomnia), invoke your API at http://localhost:5000/ to see a hello world response.

You can start seeing similarities with Express server in the simple program - more will follow.

At this stage, you may observe that any change you make will require a restart of server. There are multiple ways of solving this problem including using our dear own nodemon. Let us stick to a Go solution though - we will use a package called air to monitor for changes in our project folder and restart server whenever there are changes.

In your VSCode terminal, input -

https://github.com/cosmtrek/air

You can stop your own Go server and simply type -

air

The above command will run your server and automatically restart it whenever there are changes.

Onwards then.

(Pseudo) CRUD APIs in Fiber

Let us code in the APIs in our program.

Get Request

Introduce below line in main function before app.Listen(":5000") line.

app.Get("/todo", getTodo)

Create a new function below main.

func getTodo(c *fiber.Ctx) error {
	return c.Status(fiber.StatusOK).JSON(todos)
}

The function is (more or less) self-explanatory. The pointer c is fiber context that is injected in our function by fiber framework. We just return an ok with our todos array data. Simple enough.

Test the get API using the URL http://localhost:5000/todo and GET method from the REST client. You will see the below JSON response.

[
  {
    "id": 1,
    "name": "abc",
    "completed": false
  },
  {
    "id": 2,
    "name": "def1",
    "completed": false
  }
]

Post Request

Introduce a new line after GET request in server.go.

	app.Post("/todo", postTodo)

As it was earlier, create a new function called postTodo -

func postTodo(c *fiber.Ctx) error {
	type request struct {
		Name      string `json:"name"`
		Completed bool   `json:"completed"`
	}

	var body request

	err := c.BodyParser(&body)
	if err != nil {
		return c.Status(fiber.StatusBadRequest).JSON(fiber.Map{
			"error": "Cannot parse JSON",
		})
	}

	todo := Todo{
		ID:        len(todos) + 1,
		Name:      body.Name,
		Completed: body.Completed,
	}

	todos = append(todos, todo)

	return c.Status(fiber.StatusCreated).JSON(todo)
}

We have accomplished quite a bit, but nothing out of ordinary -

• specified how our input JSON will look like (request struct)
• included a body parser middleware to fetch input JSON data
• fetched the incoming data using todo variable
• appended todo to our todos array
• returned a success message back

The below code block takes care of error checks (in our case, this handles parse errors) -

    err := c.BodyParser(&body)
	if err != nil {
		return c.Status(fiber.StatusBadRequest).JSON(fiber.Map{
			"error": "Cannot parse JSON",
		})
	}

This type of error handling is typical to a Go program. Error checks are handled by an if statement that checks error returned from a previous statement.

Create a POST request in your REST client - http://localhost:5000/todo. Input the following JSON as input -

{
  "name": "xyz",
  "completed": true
}

Send the request and you will get the same response back. Do a GET request with the same URL and you will see the additional todo record in the response.

Of course, we are dealing only with a variable that stays in memory. The changes will disappear when the server shutsdown or restarts.

Note: the id generation logic is flaky. We used the length of the array and increment by 1. Ergo, deleting a record can lead to duplicate ids.

Delete Request

You know the drill by now.

Introduce a new REST service in main..

	app.Delete("/todo/:id", deleteTodo)

Create a new function..

func deleteTodo(c *fiber.Ctx) error {
	paramID := c.Params("id")
	id, err := strconv.Atoi(paramID)
	if err != nil {
		return c.Status(fiber.StatusBadRequest).JSON(fiber.Map{
			"error": "Invalid id.",
		})
	}

	for i, todo := range todos {
		if todo.ID == id {
			todos = append(todos[0:i], todos[i+1:]...)
			return c.Status(fiber.StatusOK).JSON(todo)
		}
	}

	return c.Status(fiber.StatusNotFound).JSON(fiber.Map{"error": "Record not found"})
}

We have used a new function for string type conversion. Include the package in the import statement.

import (
	"strconv"

	"github.com/gofiber/fiber/v2"
	"github.com/gofiber/fiber/v2/middleware/logger"
)

A couple of things to take note -

1. We are not manipulating the array directly since the array itself but creating a new array each time there’s a change with this line todos = append(todos[0:i], todos[i+1:]...). Not effective, but works. Instead we could have used a pointer and updated array directly.
2. We are using a convenient shortcut provided by fiber to create response JSON on the go - fiber.Map{"error": "Record not found"}

Patch Request

Patch is similar to delete - except that we change an element instead of removing it.

Include a new function call in main.

	app.Patch("/todo/:id", patchTodo)

Create the function.

func patchTodo(c *fiber.Ctx) error {
	type request struct {
		Name      string `json:"name"`
		Completed bool   `json:"completed"`
	}
	var body request

	err := c.BodyParser(&body)
	if err != nil {
		return c.Status(fiber.StatusBadRequest).JSON(fiber.Map{
			"error": "Cannot parse JSON",
		})
	}

	paramID := c.Params("id")
	id, err := strconv.Atoi(paramID)
	if err != nil {
		return c.Status(fiber.StatusBadRequest).JSON(fiber.Map{
			"error": "Invalid id.",
		})
	}

	for i, todo := range todos {
		if todo.ID == id {
			todos[i] = Todo{
				ID:        id,
				Name:      body.Name,
				Completed: body.Completed,
			}
			return c.Status(fiber.StatusOK).JSON(todos[i])
		}
	}

	return c.Status(fiber.StatusNotFound).JSON(fiber.Map{"error": "Record not found"})
}

All Code in One Place

We have created a program using Fiber to perform CRUD operations, but on a local variable. With a couple of more packages and a few lines of code we could connect to a DB and get to a more real-world program.

The below code represents the final state. You may also refer to this Git repo for the full code.

package main

import (
	"strconv"

	"github.com/gofiber/fiber/v2"
	"github.com/gofiber/fiber/v2/middleware/logger"
)

// Todo struct!
type Todo struct {
	ID        int    `json:"id"`
	Name      string `json:"name"`
	Completed bool   `json:"completed"`
}

var todos = []Todo{
	{ID: 1, Name: "abc", Completed: false},
	{ID: 2, Name: "def", Completed: true},
}

func main() {

	app := fiber.New()
	app.Use(logger.New())

	app.Get("/", func(c *fiber.Ctx) error {
		return c.SendString("Hello, World!")
	})

	app.Get("/todo", getTodo)
	app.Post("/todo", postTodo)
	app.Get("/todo/:id", getSingleTodo)
	app.Delete("/todo/:id", deleteTodo)
	app.Patch("/todo/:id", patchTodo)

	app.Listen(":5000")
}

func getTodo(c *fiber.Ctx) error {
	return c.Status(fiber.StatusOK).JSON(todos)
}

func getSingleTodo(c *fiber.Ctx) error {
	paramID := c.Params("id")
	id, err := strconv.Atoi(paramID)
	if err != nil {
		return c.Status(fiber.StatusBadRequest).JSON(fiber.Map{
			"error": "ID invalid.",
		})
	}

	for _, todo := range todos {
		if todo.ID == id {
			return c.Status(fiber.StatusFound).JSON(todo)
		}
	}

	return c.Status(fiber.StatusNotFound).JSON(fiber.Map{"error": "Record not found"})
}

func deleteTodo(c *fiber.Ctx) error {
	paramID := c.Params("id")
	id, err := strconv.Atoi(paramID)
	if err != nil {
		return c.Status(fiber.StatusBadRequest).JSON(fiber.Map{
			"error": "Invalid id.",
		})
	}

	for i, todo := range todos {
		if todo.ID == id {
			todos = append(todos[0:i], todos[i+1:]...)
			return c.Status(fiber.StatusOK).JSON(todo)
		}
	}

	return c.Status(fiber.StatusNotFound).JSON(fiber.Map{"error": "Record not found"})
}

func postTodo(c *fiber.Ctx) error {
	type request struct {
		Name      string `json:"name"`
		Completed bool   `json:"completed"`
	}

	var body request

	err := c.BodyParser(&body)
	if err != nil {
		return c.Status(fiber.StatusBadRequest).JSON(fiber.Map{
			"error": "Cannot parse JSON",
		})
	}

	todo := Todo{
		ID:        len(todos) + 1,
		Name:      body.Name,
		Completed: body.Completed,
	}

	todos = append(todos, todo)

	return c.Status(fiber.StatusCreated).JSON(todo)
}

func patchTodo(c *fiber.Ctx) error {
	type request struct {
		Name      string `json:"name"`
		Completed bool   `json:"completed"`
	}
	var body request

	err := c.BodyParser(&body)
	if err != nil {
		return c.Status(fiber.StatusBadRequest).JSON(fiber.Map{
			"error": "Cannot parse JSON",
		})
	}

	paramID := c.Params("id")
	id, err := strconv.Atoi(paramID)
	if err != nil {
		return c.Status(fiber.StatusBadRequest).JSON(fiber.Map{
			"error": "Invalid id.",
		})
	}

	for i, todo := range todos {
		if todo.ID == id {
			todos[i] = Todo{
				ID:        id,
				Name:      body.Name,
				Completed: body.Completed,
			}
			return c.Status(fiber.StatusOK).JSON(todos[i])
		}
	}

	return c.Status(fiber.StatusNotFound).JSON(fiber.Map{"error": "Record not found"})
}

Now go create your magic. Enjoy Go!

Javascript is the most used language in the world and that status will not change in a hurry - thanks to its frontend nature. NodeJS is super useful since we can use the same language for server and stand up a production grade server using frameworks like ExpressJS or Fastify in a matter of hours. I absolutely am in love with NodeJS -

1. Light weight
2. Easy to create, deploy and maintain applications
3. Many, many packages that make it easy to add new functionality (yes, I love them packages)
4. Decent performance

However, there are a few concerns -

1. Single threaded nature can be a cause for concern at some level. Yes, there are promises but with due respect to them - there are going to be transactions that can still make the engine “stuck”. CPU-bound, high performance tasks can be a problem in Node
2. Treatment of numbers as floats are perceived to be harmful for applications that treat such transactions as sacred
3. Types help in development-time and runtime validations that avoid all that additional code checks. Medium to high complexity projects can be (arguably?) easier to maintain with typed languages.

I am a small-scale developer + entrepreneur who creates a number of SaaS applications and also builds applications for my clients. I do not quite share any of the outlined concerns of Javascript, but I was actively looking at an alternate “fast to develop” platform about an year back. That led me to do some quick research by developing “to do” apps like there was no tomorrow. Here’s a highly opinionated view of a small-scale entrepreneur and developer on how Golang can be super useful as part of your tool-set.

Aside: While I use Typescript in many projects, I cannot firmly switch over to that camp. I love the “type less” nature of JS and it keeps my productivity high for smaller projects. Also, I found it easier to switch all the way to C# or Go.

The Alternatives

The alternatives I evaluated were -

1. Rust: Seems to be everyone’s darling nowadays, but is hard to learn and is certainly not a “highly productive” language for something like web development
2. ASP.NET and C#: I love the way C# has been developed over the past many years. I have used C# only for amateurish desktop applications, and am not afraid of extending it to web. Web assembly support and Blazor make this an exciting space
3. Golang: Simple to learn, code and deploy (thanks to the highly touted “single executable” nature of Go). Really light-weight and quite exciting async programming experience

Finally what drew me to use Go was the “promise” of easy async programming using go routines and the promise of better & consistent performance. Though I doubt whether I can put either of these to active / practical use and see exponential gains for any of my applications in the near term.

Advantage Go for web applications

Here’s a quick take on why you should consider Go in your toolset for web development.

Simplicity

Go lang is easy to learn.

Despite the strictly typed nature of Go and more complex concepts like channels, coroutines, et. al, one can easily start identifying everything that a given program is trying to do quite easily. I found this easy to pick up nature quite invaluable when trying to evaluate the complexity of web applications which were created by others but I had to go in and make changes.

Light weight

Yes, Go is as light-weight as they come despite being “decently productive”.

I can run more production applications, even more POCs and then some on a single VPS without any problems. While I did not quite have any issues per se with NodeJS based apps, I would love to imagine a not-so-distant future where I can deploy hundreds of smaller applications without spending a fortune using Go.

Performance

I am not going to lie. Despite reading up quite a bit and finding for myself on how performance of applications can be specific to a given use case, the promise of high performance in Go applications is a strong one.

After looking at Techempower benchmarks for a few years, I had resigned to the fact that my favourite frameworks are not going to be burning rubber on the road anytime soon. Go (and to a certain extent C#) is one of the ways of “future-proofing” my products and re-assure to myself that I can build “fast applications” for thousands of users.

Do note -

• often times, performance is a concept highly specific to your use case
• using a language does not ensure automatic high-performance, but the language features like async programming, “ability to make the machine work to its fullest” help. Compiled languages are often better here
• Web applications often do not have the same issues as their desktop / mobile counterparts. For e.g. Fastify + Postgres ranks higher (#148) as compared to ASP.NET + Entityframework Core + Postgres (#149!). Using an ORM in Fastify can of course throw a spanner in these results. A highly productive framework like Nest + Fastify + Mysql is not far behind (#189)

* Performance rankings are good as of 2020

Scalability

Go has lower memory footprint. It also compiles to a specific platform and occupies lower space (the latter is not quite an evaluation factor in this day and age). For me low memory usage translates to better scalability.

Concurrency and the ease of using async programming also contributes to better scalability. You could potentially use Go routines to start hundreds or thousands of threads on a 1 GB VPS without issues.

Developer-friendly

Go is easy to learn, comparatively easy to develop, and easy to deploy. Go also features in-built code formatting rules, good support in code editors and a good packaging system.

The Bad of Golang

While I would love to use Go for command line applications, I am not too sure I will switch over my web apps really soon.

1. There’s a lot of typing/copy-paste involved. Go’s simplicity also means fewer functions that provide good abstractions for common tasks. While I would love to use for loops for every problem, Javascript and C# have really good shortcuts to solve problems that are too elegant and productive to resist
2. I don’t particularly like the error handling (if err != nil). I grew up thinking the world of try/catch - so this may be an “old-man problem”
3. Golang was blessed on us a while back, but there are far few gigs on Go. SMBs stay away from the Go world since they don’t see enough developers available (= expensive to create and maintain). This is a problem for my “develop everything despite severely limited resource” mentality

I could not avoid a direct performance to NodeJS and the thousands of frameworks therein.

• Node was more productive - thanks to Javascript, and many useful packages in the Node ecosystem
• Node frameworks offer “more”. I can completely switch to Typescript and code like Angular in NestJS, or go kaput with my own structure using Express/Fastify
• Examples, demos and learning resources are bountiful in NodeJS. For e.g. integrating your favourite payment gateway is made easier with examples and possibly a SDK for NodeJS, not so for Go (though you could use their APIs as easily)

Conclusion

• Learn Go
• Use Go for side projects
• Don’t completely switch over (unless of course if you are in a Go-friendly company)

Are you sold and want to try out Go?

• Start with the “tour of Go”
• Once you are ready, start with Fiber framework. While frameworks are not as important in Go as in Node, Fiber makes transition easy since it follows the same concepts as ExpressJS

Also see: start on golang with Fiber.

But, why?

Express is the most popular server-side framework and my “go to” choice for creating anything really quick. Building a front-end for Express is as easy as using handlebars (or anything really) that goes in HTML served by Express. But that lacks a “nice” user experience.

AlpineJS is a small and sweet framework that will go places. It follows Vue in many respects (and tries to achieve parity in a lot of respects), but does not want to be a full-scaled framework. Rather, it sits happily with other code to provide user interactivity to your front-end.

Depending on what you are set to do, AlpineJS can be a good choice for your back-end application.

Use Case: Shorten URL

The premise is simple.

1. User supplies URL (e.g. http://twitter.com/techformist)
2. We shorten it and give it back to user (e.g. https://go.co/tf, assuming go.co is our domain)
3. Anyone can use the short URL. The request just bounces off our server to navigate to the longer web address

For simplicity we will ignore any authentication requirements.

We will use ExpressJS as the backend because that is one of the quicker and nicer frameworks to work with. We will use SQLite as the database.

Create a Basic Express App with SQLite DB

We will start building with some basic code. No generators, no fluff.

Create a new folder for your project and initialise using npm.

mkdir shortu
cd shortu
npm init -y

npm init will create package.json in your project folder. Edit the file to include scripts to run your application.

{
  //  ...

  "scripts": {
    "run": "node server",
    "dev": "nodemon server"
  }

  // ...
}

Install express, sqlite3 and a few useful libraries for security.

npm i --save express express cors helmet sqlite3 yup

Install nodemon so that our express application gets restarted automatically every time there is a change. This is useful during development.

npm i --save-dev nodemon

Create a new file server.js - this will be our main Express file. Let us bring in express and friends.

const express = require("express");
const cors = require("cors");
const helmet = require("helmet");
const yup = require("yup");

Include logic to initialise database.

const sqlite3 = require("sqlite3");
const db = new sqlite3.Database("./db/data.sqlite", (err) => {
  if (err) throw err;
  console.log("Connected to the SQLite database.");

  db.run(
    "CREATE TABLE IF NOT EXISTS urls (slug VARCHAR(100) PRIMARY KEY, url VARCHAR(255), clicks INTEGER)",
    (res, err) => {
      if (err) next(err);
    }
  );
});

SQLite is just a file at heart (which database isn’t?). The above logic checks for a file called db/data.sqlite. If the file does not exist, SQLite takes care of creating a new file. db.run is used to run a SQL statement.

We will execute an SQL to create a table called urls if it doesn’t already exist. There are three columns in this table -

• url - long form URL supplied by user
• slug - key part of short-form URL. Can be supplied by user or we generate one
• clicks - no. of clicks on the short URL

Initialise express.

const app = express();

// middleware
app.use(helmet());
app.use(cors());
app.use(express.json());
app.use(express.static("./public"));

// listener
app.listen(3000, () => {
  console.log("Listening on port 3000.");
});

Include a test statement to show a message for a request.

app.post("/*", (req, res) => {
  res.json({ message: "Hello, world" });
});

.. and an error function to process any errors anywhere.

// error handling
app.use((err, req, res, next) => {
  res.status(500).json({ message: err.message });
});

The basic Express application is now ready!

You can run the application using -

npm run dev

This should print a message - Listening on port 3000.

Send a POST request using your favourite API testing tool to see "Hello, world" message in response.

Define APIs in your Express Application

Before we go ahead with the API request/response, let use define a valid structure for the expected input.

// schema defn.
const schema = yup.object().shape({
  url: yup.string().url().required(),
  slug: yup.string(),
});

We will have three services -

1. Create new “short URL”
2. Navigate user to the long URL when a valid short URL is provided
3. List existing short URLs

This is also a good time to remember that we need to generate a random string if user does not provide the “slug” for the short URL. This can be done in a number of ways.

We will use a package called crypto-random-string. Let us install the package.

npm i --save crypto-random-string

Alternatively, you can use -

• nanoid
• randomstring

Let us start coding in the services.

Create new “short URL”

Create a new short URL for the user-provided URL.

1. The generated short URL will be the combination of our domain (where the Express application is running) + a short slug.
2. Slug is provided in request. It is optional, generate a URL-safe slug if not provided
3. Store long form URL and the slug in the database

app.post("/new", async (req, res, next) => {
  // get url and slug from input
  let { url, slug } = req.body;
  console.log("req.body: ", req.body);

  try {
    await schema.validate({ url, slug });

    if (!slug) {
      // generate slug if not provided by user
      const cryptoRandomString = require("crypto-random-string");
      slug = cryptoRandomString({ length: 10, type: "url-safe" }).toLowerCase();
    }

    // insert everything in DB
    db.exec(
      `INSERT INTO urls(url, slug , clicks) VALUES('${url}', '${slug}', 0)`,
      (err) => {
        console.log("err: ", err);
        if (err) next(err);

        // response message has the short URL
        res.json({
          shorturl: `http://${req.headers.host}/${slug}`,
          slug: slug,
          url: url,
        });
      }
    );
  } catch (e) {
    next(e);
  }
});

It is time to test this out!

Send a request to http://localhost:3000/new like so -

{
  "url": "http://google.com",
  "slug": "ro0wgfvwxg"
}

You should receive this beautiful response -

{
  "url": "http://google.com",
  "slug": "ro0wgfvwxg",
  "shorturl": "http://localhost:3000/ro0wgfvwxg"
}

Your short URL generation is ready, onwards to doing something with this request.

Navigate user to long URL

With the previous service the user received this short URL http://localhost:3000/ro0wgfvwxg, which can now be happily shared by her to thousands of people.

When these people use the URL in a browser, the request is sent to our server. This can be fulfilled by a service that will fetch the long-form URL from the given short URL, and automatically navigate users to the correct destination.

This is easier than it looks.

app.get("/:id", (req, res) => {
  // id = slug. e.g. `ro0wgfvwxg`
  // fetch data from DB

  db.get(`SELECT * FROM urls where slug='${req.params.id}'`, (err, data) => {
    // navigate user to the right URL
    res.redirect(data.url);
  });
});

The above code should do the trick. But, let’s improve that a bit more -

• add error handling - navigate user to an error page if slug is not found
• add logic to count number of clicks. Each time anyone uses the URL, the number of clicks will be incremented

app.get("/:id", (req, res) => {
  db.get(`SELECT * FROM urls where slug='${req.params.id}'`, (err, data) => {
    if (err) next(err);
    if (!data) res.redirect(`/?error=${req.params.id} not found!`);
    else res.redirect(data.url);

    // add clicks
    db.exec(
      `UPDATE urls SET clicks = clicks + 1 WHERE slug='${req.params.id}'`,
      (err) => {
        if (err) console.error(err);
      }
    );
  });
});

Now, try navigating to https://localhost:/ro0wgfvwxg in the browser. You should be navigated to the correct URL http://google.com.

List URLs

Let us code the last of the services - list all URLs in the database.

This service is similar to the previous one, except that we run a simpler query to fetch all URLs and return them to the caller.

app.get("/list", (req, res) => {
  db.all(`SELECT * FROM urls`, (err, data) => {
    if (err) next(err);

    res.json({
      data: data,
    });
  });
});

Try it! A request to https://localhost:3000/list should return all the URLs from the database.

{
  "data": [
    {
      "slug": "ro0wgfvwxg",
      "url": "http://google.com",
      "clicks": 3
    },
    {
      "slug": "whphznmvcf",
      "url": "http://bing.com",
      "clicks": 0
    }
  ]
}

Your backend application is now fully ready to take on the world.

Front-end for Shorten URL Application

Create a simple HTML file index.html in the project root, which will contain the whole of our frontend. Include AlpineJS from CDN.

<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=400px, initial-scale=1.0" />
    <title>Shortu</title>

    <script
      src="https://cdn.jsdelivr.net/gh/alpinejs/alpine@v2.x.x/dist/alpine.min.js"
      defer
    ></script>
  </head>
  <body>
    Frontend Does Not Rock
  </body>
</html>

Let us code some CSS since no one likes to see default styling. Since I am too lazy to do the complete styling by myself (capability aside), I will use a simple CSS framework called “MVP.css”.

Include MVP.css and a custom CSS (for minimal style changes) in the HTML head.

<link rel="stylesheet" href="https://unpkg.com/mvp.css" />
<link rel="stylesheet" href="./assets/custom.css" />

Create folders in the project root.

mkdir public && cd public && mkdir assets
cd assets
touch custom.css

We can use powerful functions to bind HTML elements to backend. Let us create a form and the necessary objects that will act as the container for all functionality.

<body x-data="process()">
  <h1 class="title">Shortu URL Shortener</h1>
  <div class="subtitle">
    Input the long URL, hit "Shorten" and see magic happen.
  </div>

  <form class="input-group">
    <input type="text" placeholder="Provide the long URL" x-model="url" />
      

    <input
      type="text"
      class="form-control"
      placeholder="Desired short url phrase (optional)"
      x-model="slug"
    />
  </form>

  <script>
    function process() {
      return {
        url: "",
        slug: "",
      };
    }
  </script>
</body>

Note that -

• process() will provide the infrastructure for AlpineJS. You can define JS variables, functions and more within process function
• x-model binds the HTML element to the backend variable (for e.g. url). This is similar to v-model in Vue

Express serves the HTML file by default when we navigate to https://localhost:3000/. Navigate to this URL in the browser to see your application in action.

Let us add functionality to allow user to send request to Express and see results.

Add a button that sends data to backend Express, and elements to show results (or error). Input below code before the ending </form> element.

<button x-on:click.prevent="shortenURL()">Submit</button>
<div x-show="error" x-on:click=" error = ''">
  <span x-text="error" class="error"></span>❌
</div>
<div id="results" x-show="data['shorturl']" class="results">
  <div class="label">Your short URL:</div>
  <h3 style="display: inline-block;" x-text="data.shorturl"></h3>
</div>

Add the corresponding functions and variables in process.

• data to store the response URL list and error to store error
• shortenURL - a function that invokes our new service to shorten a given URL

The complete <script> element is below -

<script>
  function process() {
    return {
      error: "",
      data: {},
      url: "",
      slug: "",
      shortenURL: async function () {
        try {
          console.log("this.url", this.url);
          const result = await fetch(`/new/`, {
            method: "POST",

            headers: {
              Accept: "application/json",
              "Content-Type": "application/json",
            },
            body: JSON.stringify({ url: this.url, slug: this.slug }),
          });

          const resJson = await result.json();
          console.log("resJson: ", resJson);

          if (result.ok) {
            console.log("result.ok: ", result.ok);

            this.data = resJson;
            this.error = "";
          } else {
            console.log("in error");
            this.data = {};
            this.error = resJson["message"];
          }
          console.log("data: ", this.data);
          console.log("error: ", this.error);
        } catch (e) {
          console.log("error", e);
          this.error = e.message;
        }
      },
    };
  }
</script>

Voila, your application is ready just like that.

Complete code is at this Github repo.

The End

Building applications with Express is fun, and libraries like Alpine make the user experience fun too. I have to say - I am quite smitten this smart library.

The Art of Wordpress Site Migration

It may be just me but I don’t quite understand why Wordpress site migration has to be so “difficult” for a typical user. Note that I said “difficult” as compared to other user-facing, ‘citizen-admin friendly’ platforms. It is indeed cake-walk when compared to typical developer-owned tasks. I may not exactly an expert to deep dive and show why things are as they are but, there are more than few ways of making the process easy.

Before we go further on the topic, let’s look at typical use cases for migrating Wordpress sites -

1. Migrate sites from one host to the other The movement of sites from one host to the other result in different database names, connection parameters, and file structures.
2. Migrate sites from one domain to the other This refers to moving site from one domain (say example.com) to a new domain (newexample.com), or migrating from a test to a production system (example.test to example.com).

We don’t have cover all use cases to keep this post sane, instead we focus on “typical” migration cases that one (me!) is most likely to do.

Here are the typical questions when migrating sites -

1. Which components need to be migrated?
2. How should we move data/files/other?
3. What options do I have for Wordpress site migration, and which is the best option?

I fussed with such questions over years, used different workflows, and finally, developed one plugin-independent flow that simply works. The workflow may not be optimal (let me know!), or may be even wrong - but it does get the work done. So, YMMV.

This flow is more geared towards people who are comfortable working with databases, applications and so forth. Here are the pre-requisites -

• Database created in the target environment. User id /password and relevant host details are available to you
• File system access (e.g. through an explorer in CPanel) available to you

Components for Migration

Wordpress is based on PHP, and like many PHP applications, is quite simple at heart. We primarily deal with two components -

1. Files
2. Database

Files

Files reside in wp-content folder within Wordpress root installation folder. They may include -

1. User data including images, video and other media, data files that you may be using in your site, files made available for download, and other user generated files.
2. Site customisation files - themes, styles, etc.

Database

Export database from the source and import in the target database.

You can change the database to reflect the new site in a one of the below ways -

1. Change strings in export file before importing content in target
2. Run SQL in target database after the import
3. Run a PHP script that will do the updates for you

Workflow for Wordpress Site Migration

This workflow is generic in nature and works for typical migration use cases -

1. Moving sites from one host to the other
2. Moving sites across environments (e.g. from test to production)

Copy Files

All you are expected to do is copy the entire Wordpress folder from the source to the destination. You could also copy/paste wp-content on the same versions of Wordpress with similar configuration.

Once the files are copied over, you do a few changes -

1. Update configuration (for e.g. in wp-config) to point to the right database. Your old file will have the older database details. Change -

   • Database host and/or database name (CPanel may enforce naming convention of prefixing table names with your root domain. e.g. techformist_wp_options)
   • User id / password

2. Revert .htaccess file in the root folder to the default. You can also regenerate this file from within Wordpress. In its simplest form, .htaccess looks like the below.

   # BEGIN WordPress
   
   RewriteEngine On
   RewriteBase /
   RewriteRule ^index\.php$ - [L]
   RewriteCond %{REQUEST_FILENAME} !-f
   RewriteCond %{REQUEST_FILENAME} !-d
   RewriteRule . /index.php [L]
   
   # END WordPress
   

Export and import database

We do this database change to migrate Wordpress sites across domains. If you have the exact same domain (with HTTPS) - skip this step.

Login to your source environment. Export database to a file in source.

If you are using CPanel, go to phpMyAdmin from your control panel home. Select your database and go to Export tab.

Defaults will work for most cases, just click Go to export database to a file. This will export the table structure and data.

If you are migrating to an existing database and want to overwrite target with data from the source, explore the options to delete and re-create tables -

• select the Export method as Custom - display all possible options
• then select Add DROP DATABASE IF EXISTS statement option.

You can import the file in target database as-is, but it may make sense to change the export file before you import. This is particularly useful if you are changing your domain name or migrating from test to production (also a domain name change).

1. Open the file in an editor like Notepad++
2. Hit Find and Replace
3. Find for all instances of oldurl and replace them with newurl
4. If the new url is not setup yet, it is a good idea to replace with http version of the URL rather than using https

To import the file -

1. Logon to your phpMyAdmin on target host
2. Navigate to Import tab
3. Select the file to import and hit Go

[Optional] Updating database without file modifications

Again, this step is relevant only if you are changing domain names.

If you cannot / do not want to work with files, you can always work with SQL to update database to work in your new setup.

First, import the files in target database using phpMyAdmin or similar tools.

In phpMyAdmin, go to SQL tab. Paste the following SQLs .

UPDATE wp_options SET option_value = replace(option_value, 'oldurl', 'newurl') WHERE option_name = 'home' OR option_name = 'siteurl';
UPDATE wp_posts SET guid = replace(guid, 'oldurl','newurl');
UPDATE wp_posts SET post_content = replace(post_content, 'oldurl', 'newurl');
COMMIT;

Replace oldurl and newurl with your relevant URLs from the source and target systems. Hit Go.

Finishing up

Provision SSL certificates for your target site. Once again, this is made easy by control panels.

In CPanel -

1. Go to Let's Encrypt link on the home page of control panel
2. Select Install Certificate against your domain

In a VPS just install Certbot, and it will do the rest for you :).

Update your site to use https-

1. Login to <yourdomain>/wp-admin
2. Go to Settings > Select General
3. Change WordPress Address (URL) and Site Address (URL) to use https. Click on Save Changes. This will throw an error message that you can disregard
4. Re-login to wp-admin using the new https URL. Go to Settings > Select Permalink from menu
5. Under Common Settings change the URL setting to any option and revert back to your desired option

Alternatives

The workflow works for me because I am not afraid of databases and servers. I also do not work on Wordpress actively and may be do < 5 migrations per year.

There are alternative solutions that are more elegant, but have dependencies on third party tools.

Plugins

There are plugins that you could use to automate the tasks in part or fully.

1. Updraft Plus allows you to backup database and files from the source. You can “restore” the content in the destination. Updraft plus handles many complexities including updating the new host names, etc.
2. WP Migrate DB takes care of exporting databases while allowing you to find and replace content within the database. The pro version enables you to work with files as well

Despite using plugins, you still have to do the backend configuration work including -

1. Updating wp-config to point to the new database and new file system (you may skip this step if you use Wordpress installation done by someone else or by one-click installers)
2. .htaccess updates

CPanel Migration

When you are on CPanel and changing hosts, check with your new hosting company whether they support free migrations. CPanel allows your hosting company to transfer from an existing host without any fuss - this is an one-time migration.

The process requires no effort from your end. Just buy new hosting, open a ticket for migration, and you are done. Your site will continue to function without problems.

Similar solutions may exist for other control panels too.

You are done!

Enjoy your new site.

1. Get a cheapo VPS on Digital Ocean, Vultr (get $100 credit), Hetzner, OVH, and friends - I end up choosing Ubuntu as the OS
2. Secure your server
3. Setup app server, database server and tools
4. Stitch together various components through configuration files

I have thought about switching to Docker (or Caprover) for months, but keep putting off any kind of automation. The adhoc sharing of DB servers, file systems, report/analytics servers make it a tricky exercise. I don’t quite spend more than 4 hours on a crazy day - so it’s not quite a big deal.

Not being a sys admin myself I follow a few “best practice” steps recommended by smarter people on the Internet. Here’s an easy-to-follow list for setting up a secure VPS.

1. Get Access and Logon

I assume that you have -

1. Already created the server using the browser console provided by your favourite service provider
2. You have chosen Ubuntu LTS (v20.4 at this time, though it should quite happily work the same for v18.4)
3. I use a Windows computer to connect to VPS. A few steps for configuring client will differ if you are using Linux or Mac

Login to your server.

ssh root@<password_in_email>@<server_ip>

You will get a message like this -

The authenticity of host '11.11.111.111 (11.11.111.111)' can't be established.
ECDSA key fingerprint is SHA256:XXBLAH.
Are you sure you want to continue connecting (yes/no)? yes

Enter yes to continue.

You can make this step more secure when creating servers with some providers like Hetzner. You can enter the SSH public key at the time of server creation and get seamless access to the server once it is created. We will get to the public key part soon.

Change root password.

passwd

Enter new password for root. It is a good idea to store this in LastPass or some other password management tool.

Add a new admin user. This id will be the default to login to your server.

adduser aweadmin

Enter password for new user. Next allow the new user to sudo.

usermod -aG sudo aweadmin

The goal is to get farther from defaults to make it more safe. This by itself will not completely secure the server, but reduce the risk of your server being of any interest to hackers.

2. Enable Automatic Security Updates

First, let us update the OS.

apt update
apt dist-upgrade

Once the update is complete, reboot server.

reboot

Your SSH session will disconnect and server will reboot. Re-login using SSH to your server.

Recent versions of Ubuntu have unattended updates by default. You can verify or change those updates -

sudo nano /etc/apt/apt.conf.d/50unattended-upgrades

If you are using an older version, you may need to install and configure automatic updates.

apt install unattended-upgrades
nano /etc/apt/apt.conf.d/50unattended-upgrades

Comment out everything other than the security updates.

Unattended-Upgrade::Allowed-Origins {
//      "${distro_id}:${distro_codename}";
        "${distro_id}:${distro_codename}-security";
        // Extended Security Maintenance; doesn't necessarily exist for
        // every release and this system may not have it installed, but if
        // available, the policy for updates is such that unattended-upgrades
        // should also install from here by default.
        "${distro_id}ESM:${distro_codename}";
//      "${distro_id}:${distro_codename}-updates";
//      "${distro_id}:${distro_codename}-proposed";
//      "${distro_id}:${distro_codename}-backports";
};

Save file with Ctrl+O > Enter. Exit the editor with Ctrl+X.

So far you have configured automatic updates. Next, enable the automatic updates.

nano /etc/apt/apt.conf.d/20auto-upgrades

Copy/paste the following lines -

APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Download-Upgradeable-Packages "1";
APT::Periodic::AutocleanInterval "7";
APT::Periodic::Unattended-Upgrade "1";

3. Change SSH Port and Default User Access

We will do two things here -

1. Change SSH port & protocol
2. Enable additional SSH user and disable root access

Change Port and Use Protocol 2

SSH uses port 22 by default. But, you can choose any free port between 1023 and 65535. You can find out whether a port is free with the following command -

ss -tulpn | grep LISTEN

Open the config file and change the default 22 port used by SSH.

nano /etc/ssh/sshd_config

Find the line which begins with Port and change it -

Port 65432

While we are at it, let us also add a line to force SSH to always use Protocol 2. Just add the below line anywhere in the file.

Protocol 2

Ctrl+o > Enter + Ctrl+x to save and exit out of the editor.

Restart SSH.

sudo systemctl restart ssh

Enable non-root SSH user

Open another command prompt. Try logging in to your server with the user id that was created earlier.

ssh -p 65432 aweadmin@11.11.111.111

Once you are in, try to run a sudo command.

sudo ls /root

If things worked well so far, it is time to disable root login using SSH.

Open the SSH config on server.

sudo nano /etc/ssh/sshd_config

Change the below parameter to no and insert a new line to enable access for the new admin user.

PermitRootLogin no
AllowUsers aweadmin

Ctrl+o > Enter + Ctrl+x to save and exit out of the editor.

Restart SSH.

sudo systemctl restart ssh

Open yet another command prompt and try logging in as root.

ssh -p 65432 root@11.11.111.111

This should not work but aweadmin will login without problems.

4. Disable SSH login with password

If you ever check the security logs on your server, you will find a series of attempts to access your server with default user ids, ports and common passwords. In this section, we will disable SSH access using password altogether and enable only clients with trusted keys to login.

Let us first set up the client (which is our computer = not server).

Run following command to generate a public/private key on your computer.

ssh-keygen -N "securepassphrase" -t ed25519 -C "mycomputer"

• securepassphrase - can be anything that you can remember but at the same time secures access to SSH server from your computer
• ed25519 provides the encryption algorithm used to generate keys
• mycomputer is just a random name which is a description for the key. You may want to provide different descriptions on other computers

Accept the default file names for public and private keys (or change them). You should have two files in c:\user\<user_name>\.ssh\ folder.

• id_ed25519 - file with private key
• id_ed25519.pub - public key

Next, we update the generated public key on the server.

ssh -p 65432 aweadmin@11.11.111.111

Create .ssh directory (if it doesn’t exist) and create a new file within in.

mkdir /home/aweadmin/.ssh
nano /home/aweadmin/.ssh/authorized_keys

Copy / paste the public key from id_ed25519.pub.

Ctrl+o > Enter + Ctrl+x to save and exit out of the editor.

Change the default permission for the newly created file -

chmod 0600 /home/aweadmin/.ssh/authorized_keys

Open a command prompt on client, and login with key.

ssh aweadmin@11.11.11.1111

You will get a different prompt that asks for the key passphrase. Enter the passphrase to gain access to the server - you are not required to type the aweadmin’s password again.

Next, we will disable password logins in the SSH config file -

sudo nano /etc/ssh/sshd_config

Change following parameter -

PasswordAuthentication no

Ctrl+o > Enter + Ctrl+x to save and exit out of the editor.

You are required to enter passphrase when logging in each time. We can work around that.

Go to Start > Services to list Windows services (or Start > Run > Enter services.msc). You should find a service called OpenSSH Authentication Agent service that will be in disabled state by default. Enable the service to start automatically with Windows and start the service.

Go to Start > Run > type notepad c:\<user>\.ssh\config. Enter following lines -

Host 11.11.111.111
  User aweadmin
  IdentityFile ~/.ssh/id_ed25519
  IdentitiesOnly yes
  Port 65432

Save file. Ensure that the file is saved without the default txt extension.

You should now be able to login to the server without entering user id, password or key passphrase.

ssh 11.11.111.111

If you use multiple computers, or want to provide access to other users, just repeat the steps to generate distinct private keys and update public key on the server.

5. Secure Your Server

Let us install and enable software to secure the server.

Firewall

I typically use UFW because it is quite simple to setup and understand.

You can install / update ufw with -

sudo apt install ufw

Check status by using -

sudo ufw status

Configure ufw to allow SSH sessions on our SSH port. This is important since ufw closes all ports by default. Enable firewall.

sudo ufw allow 65432/tcp
sudo ufw enable

Open another command prompt and try accessing the server with default port - you should be able to login.

You can check the rules with the command -

sudo ufw show added

Intrusion Detection

We will install Fail2Ban to work alongside the firewall and secure our server. fail2ban is an intrusion detection and prevention framework that can automatically ban IPs temporarily (or otherwise) based on invalid login attempts.

sudo add-apt-repository universe
sudo apt-update
sudo apt install fail2ban

Copy the default config file to jail.local and restart service.

sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
service fail2ban restart

6. Get on with your apps

With your server setup you can start installing your applications. A few of those are super useful -

1. Vim Editor

   sudo apt-get install vim
   

2. Install fish as default shell, and omf to configure fish

   apt-get install fish
   chsh -s /usr/bin/fish
   curl -L https://get.oh-my.fish | fish
   

3. Caddy Web server

   sudo apt-get install caddy
   

4. Install Node (v12.x)

   curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash -
   sudo apt-get install -y nodejs
   

5. Install MySQL database and configure MySQL. You will also need to open port in ufw.

   sudo apt install mysql-server
   sudo mysql_secure_installation
   

That’s all folks - enjoy your brand new VPS!

So.. ASP.net?

Yes, indeed. I have had a love-hate relationship with ASP.NET through years. I am way less productive using ASP.net but cannot ignore the speed that a dotnet web server provides. Take into account the super debugging capabilities/tooling and the sizeable market that keeps providing projects on the platform, we surely have more than a winner in ASP.NET.

And oh, it immensely helps that innovations like Blazor are exciting options to make web development arguably more productive and easier.

Why yet another introduction?

There are a million other introductions to ASP.NET platform. Microsoft’s own documentation and guides are immensely helpful for beginners at all stages. But, what I love to see is a quick way to create an app (or two) and make new developers comfortable with the ecosystem. At least that’s how I learn stuff and I found some of the tutorials I glanced through to be either too detailed, or at a superficially high level.

This introduction is more for developers who use a different language for web development. It is not quite intended for absolute beginners, but anyone should be able to follow along.

Firing off with ASP.net

There are two main ways to get started on development using ASP.net platform.

1. Use Visual Studio: proven way that has been forever
2. Use Visual Studio Code or your favourite editor and dotnet core CLI

Both ways deserve all the love they can get. I prefer VSCode, but will use Visual Studio in this tutorial because - why not. Going forward I will be basing discussions primarily on Visual Studio 2019 and dotnet Core 3.1.

Installation is somewhat boring and works on expected lines.

1. Download Visual Studio from https://visualstudio.microsoft.com/
2. Click, click, click to install

Visual Studio comes with dotnet SDK, and you should be all set by now. If you decide to use VSCode, just install the dotnet SDK separately. You should then be able to use all the command line goodness on the best code editor ever(tm).

See the install docs if you are stuck. (Stuck during install, really?)

Once everything’s ready, just create a new project.

Select “ASP.NET Core Web Application”. Hit next and name your project. You will get another screen where you can select an “Web Application” template. This option will instruct Visual Studio to scaffold a few things when the project is created.

The Structure

As any modern web development stack, the files generated can be overwhelming at the beginning. We can break the structure down to basics.

A typical ASP.net project consists of one solution (which is a container for projects), and one or more projects. It will have at least one project (of course), anything else is optional and enables you to build stuff incrementally.

Beginning of everything: Startup.cs and Program.cs

These are the two files that bootstrap our app and give it ASP powers. When our app starts, the runtime calls Main method defined in Program.cs, which configures itself and starts running as an ASP app.

If you were so far wondering how our humble app came to acquire ASP powers, IHostBuilder is to blame. That interface provides our function CreateHostBuilder abilities to assume the powers of ASP.net, configure itself and start running - all in a simple one-line statement..

CreateHostBuilder(args).Build().Run();

More code in Program.cs uses C# lambdas and Dependency Injection (“DI”) to apply configuration parameters and methods defined in Startup.cs to our application.

public static IHostBuilder CreateHostBuilder(string[] args) =>
        Host.CreateDefaultBuilder(args)
            .ConfigureWebHostDefaults(webBuilder =>
            {
                webBuilder.UseStartup<Startup>();
            });

Further, ConfigureServices in Startup.cs has a specific block that enables Razor pages in your app.

public void ConfigureServices(IServiceCollection services)
{
    services.AddRazorPages();
}

The Configure method in Startup.cs enables a pipeline to include middleware, which in-turn enables various features for your app - all without needing you to code them in.

For e.g. -

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
    // ...
    app.UseHttpsRedirection();
    // ...
}

… tells runtime to redirect HTTP requests to HTTPS.

We will deal with lambda functions, DI etc. in more detail some other time. For now, we will move on and find out how we can quickly create our app on ASP.net.

Before we dig-in: settings and launchSettings.json

We have created a web application that runs as a command line app. The server is just serving content defined in web stuff like HTML/JS/CSS etc and in Razor pages, but does not render UI on its own.

Configuration parameters for our app is defined in launchSettings.json.

You can see the values defined in settings file in a nice user interface - right-click on the project (not the solution) in the right-hand-side bar, and select properties. Any change in one place is reflected in the other - just sayin'.

You may also notice the word “development” in the configuration file. We can provide parameters for any number of environments including development, and the start-up logic figures out the settings to apply. Further, secrets can be managed quite efficiently and made accessible to those who “need to know”. This is super helpful as you might imagine.

By default, the app uses IISExpress but there are options to use Kestrel, which is a light-weight (& a more modern) server by Microsoft. You will see these configuration options selected in the Toolbar as well. Typically, I prefer Kestrel since it is fast, and I end up hosting dotnet on Linux (where IIS is not available).

The ‘client-facing’ wwwroot

All the client-facing assets are stored in the www folder. This includes CSS, JS (custom and any libraries), static HTML, and anything else that finds its way to the browser.

Right off the bat, you see a sample CSS and JS, bootstrap and JQuery in the scaffolded files. Any and all of this can be changed or replaced. For e.g. you can replace Bootstrap with Bulma and chug along a different path.

Pages folder

Most of the action takes place here. Pages is where you place razor pages created using C#. The pages in C# get converted to something that the browser can understand by ASP.NET – we will see way more of pages in a few moments. Razor pages have the extension ‘cshtml’ files

You also see a Shared folder and a bunch of files starting with _. Shared folder is used to contain assets shared across the project, and the _ is a convention to create something called a ‘partial’. A partial is not a ‘complete’ page in itself, but comes together with other partials or pages to form complete pages.

Layout.cshtml is an example of a partial. In this file we have the header (which forms the header in the pages using this layout), a container (which will contain data), and the footer. The file also has references to script and HTML.

Of course, you can use different layouts, or decide not to use layouts at all – but building pages like this will make the entire process easy to build and to maintain.

In other notes, take a look at -

• _ViewImports.cshtml: enables tag helpers globally
• _ViewStart.cshtml: base/master page

Error, Index and Privacy pages are the actual Razor pages. You will create something on these lines to for adding more pages to your app.

Expand Index.cshtml page to find a file called Index.cshtml.cs. This file has the C# code to go along with the mark-up page. You can refer to the variables, methods, etc. from this c# page in the mark-up cshtml page. This arrangement is akin to mark-up + code arrangement in any Vue/React/Svelte application. In Blazor, you can have mark-up + code in same or different files - but that is for a different day.

This structure is different from a typical MVC model that touts a distinct controller, model and view layer. These functions are present in the Razor page app, but not fashioned in MVC style.

More about the internals and workings of an ASP.net app

We are now masters of the structure of ASP.net core razor page project, and it is time to move on and see how everything comes together.

If you’re serious about your trade, you will skip this section altogether. The terms here just help you speak the same language - you will probably figure this out by yourself while building the actual app.

Routing

The Pages folder takes care of routing in our application. The files in the folder represent the URLs served by your application.

Let’s see how..

• Privacy.cshtml is a file in the Pages folder
• baseurl.com/privacy will request the page named ‘privacy.cshtml’ in the folder. ASP.net serves the HTML generated from the razor page.
• Any sub-folder will also get included in the path. For e.g. a blog/Hello.cshtml is rendered on baseurl.com/blog/Hello
• If the requested page does not exist, a ‘page cannot be found’ error is displayed by default. You can configure the application to redirect requests to any other page (e.g. index) if the page cannot be located

You can override the default routing using the @Page tag.

@page "/privacy1"

.. will enable the link to baseurl.com/privacy1 and the previously used /privacy page disappears into oblivion.

Start Tinkering

Run your app by hitting Ctrl + F5. Visual Studio opens the browser (or allows you to configure browser) and serves the application. You can see the starter page and navigate through the links provided in the scaffolded application.

You will see that any changes will require you to build/restart app. We can avoid that by executing our app in watch mode..

cd <project-folder>
dotnet run watch

Why Scrum?

If you are looking for the answer on why agile / scrum -

1. You are in the wrong site
2. That is well beyond the scope of this post

In short: Scrum is a framework to run your project using Agile methodologies. Scrum’s objectives are -

1. Self-organise
2. Learn by doing things and through experience
3. Continuously improve

Read more on Atlassian’s site.

The stated objectives for Scrum need a simple to use, centralised store - which an Excel document is generally not. But, I don’t let all that come in the way to create a beautiful template.

Download Excel Template

If you like to live on the edge and download from any website -

Download free Scrum Excel-template.

If you are not that naive, you can check out what this template can do (or even copy the workbook) on Google Sheets.

See the action on Google Docs.

The template is -

• free to use for any kind of project
• released to public domain - see the sole exception in the terms below

You can stop reading now, and get back to work in 99.99% of the cases.

Why use Excel to track Scrum and why this template?

• There are smaller teams that rely on part-time developers who don’t want overhead
• Small teams may discuss on “stuff” and move on with work, and don’t particularly care about documenting everything
• Simple projects may not have the budget to organise things

And yes, I know that these are excuses for not creating a “usable” tool. But organisations continue to live in a bubble.

So, here we are.

Using this template you can -

1. track projects and sprints
2. track backlog, impediments and retrospective remarks
3. record effort against tasks (which can relate back to backlog)
4. summarise effort and show beautiful charts

There are no fancy formulae, macros, and automation to take the dog on a walk when you work.

All that said, I think no one should use Excel templates for use cases like this. But I really can’t quite preach something else when I have been an ardent fan of their simplicity.

Terms of Use

No terms, really. You can do what you want with this template.

But, I would appreciate if you could donate a dollar or more to the nearest charity, or to a person in need.

I came across this whitepaper on testing micro services. I clicked on this link because I am writing microservices now but I am fairly new to microservices.

I did a mistake that I have repeated 1593 times in the past. The said link was a resource by one of the big IT service providers.

While the work indeed is commendable - automating 4000 test cases is no joke, the content and approach for highlighting the “best practice” leave much to be desired.

The paper highlights practices that have been assumed to be best practices for a long time - and provides absolutely no information other than to say “we did so and so”. “Extreme automation” is the buzzword of the day, and as we, of the corporate types, agree - it doesn’t get better than that.

I mean - yes, I understand this is a mere case study. I don’t expect every case study or paper to contain all the real “stuff” all the time, but following would have been welcome -

1. How does the accelerator fare as compared to commercial tools today?

• Savings in license costs by avoiding commercial tools (typically clients don’t pay or think they don’t pay for “accelerators”)
• How well did the accelerator write tests? What was the coverage advantage vis-a-vis other tools
• How did developers breathe easy by using this tool as compared to say writing tests in Postman (Pro), or using other open source and free tools that are popular today

1. By how much did the framework improve efficiency in generating data?
2. How was a high degree of overlap achieved in smart sharing of regression/system test cases, while keeping effort and test duration minimal

Take note that I have still not outlined just how I am doing the “thing”. That may be part of the IP, or reflect my “scarcity mindset”. But, readers only get excited about the work when they know more about the advantages of tools and practices employed in the project.

A more complete paper would probably also cover -

1. Employing live vs. test data, and how the framework can help in specific cases. How we found advantage in one or both?
2. How performance tests play a greater role today? How can they be integrated in the overall cycle? How are they automated using existing toolsets?
3. How well can we mock or virtualise services more efficiently given a specific set of constraints?

It’s not all bad

Don’t get me wrong. I can understand where the above content is coming from. Marketing pressures, typical development processes not covering best practices, and lack of resources to think about content strategy through and through - they are all real.

And.. I have been there multiple times. I have been guilty of providing such content, case studies and “white papers” in the past (and probably churn that out again?) - content that lacks substance for so called “expert readers” but content nevertheless. I hope the servers hosting them will burn to ground so that I can lecture on best practices of writing good content. Oh wait, I hope it does not include this website too.

Fortunately, good content is everywhere today. Want to automate your API testing and write something about it? I would say start from these posts -

• know what API testing is about - https://martinfowler.com/articles/microservice-testing/#agenda
• free and open source tools - Integrations of swagger with open source tools including SwaggerAssertions, swagger-test
• how a popular tool aids writing your tests - look up Postman Pro
• how a seemingly successful product helps test automation - https://assertible.com/blog/testing-an-api-using-swagger

The Situation

Vuepress is simple.

Being simple is rather difficult. In the case of Vuepress, the difficulty can show up in unexpected places.

Starting with Vuepress is simple enough. Equally simple is enabling navigation on your site at multiple levels through a “sidebar” and/or a “navbar”. Specifically for the sidebar - you can enable in by adding a couple of lines in /docs/.vuepress/config.js.

module.exports = {
  //...
  themeConfig: {
    sidebar: [
      { title: "Home", children: [""] },
      {
        title: "Misc",
        children: ["more"],
      },
    ],
  },
  //...
};

The above change will create a sidebar for your site with links under Home and Misc. The title of links will be either -

• the title of the page (or)
• the first header element within the markdown files that the links point to

This is how it looks -

As you can see: the H2 headers will be links grouped below the file links.

But, what if you need to do more -

1. you want to create different sidebars for different pages. For e.g. if you are creating a site that has a bunch of different guides instead of one guide with multiple chapters. When you go to specific pages (which are represented as folders in Vuepress root folder), the navigation links have to change to point to different markdown files in a specific folder
2. you may want to continue having H2 behaviour be Vuepress default - show the links within the specific markdown files
3. you don’t want to write links to all the files by hand. Instead you want the system to automatically “see” the content to be linked

In a recent project I needed the “Admin”, “Developer” and other folders to have their own sidebars with navigation links. -

This sidebar is slightly different from Vuepress default, which seemingly assumes that all of the content is one big happy family. Vuepress by default enables links just point to the different markdown files in the document root, and to the H2 elements when the parent links are expanded.

Solutions

Let’s look at how we can customise sidebar in Vuepress.

Vuepress is after all powered by Vue - so customisation is not really a complex problem. However, I did not find this as straight forward as something like Gridsome. In Gridsome you just create a component for navigation like any sane developer and the system will exactly do what it has to - render the navbar.

Vuepress’s navbar is driven by config (and if applicable, plugins) - so this seemed not so elegant(!?) at first. But as always - all you need to solve world’s problems are good docs and some time.

Given below are three solutions that I tried to different degrees of success. The solution that worked for me is at the end because I am heartless and believe in making people scroll.

Solution 1. Use vuepress-plugin-auto-sidebar plugin

The plugin vuepress-plugin-auto-sidebar seemed to do exactly what I wanted.

To install a plugin in Vuepress, all you have to do is -

1. Install the package using npm/yarn
2. Use the plugin in config

Let’s get this going. Go to Vuepress folder and install package.

npm install --save vuepress-plugin-auto-sidebar

Include plugin in /docs/.vuepress/config.js-

module.exports = {
  // ...
  plugins: {

"vuepress-plugin-auto-sidebar": {
      titleMap: {
        "salesforce-for-students": "Salesforce for Students",
        "pd1-guide": "Platform Developer I Certification Guide",
        "admin-guide": "Admin Certification Guide",
        "why-learn-salesforce": "Why Learn Salesforce?",
        misc: "Miscellany"
      }
    },
// ...
}

This configuration will provide custom titles for the different folders and automatically pick up the files within the folder to create distinct, dynamic navigation bars.

In the ideal world, this should have been it and I should not have been sitting around to create a blog post about all the circus. But hey - ideal world is not exciting.

1. Getting started was bit of a bother. The documentation of the package is in Chinese and there are not many examples. Google translation helped to an extent
2. I could not quite figure out how autoNext and autoPrev worked. My files were not exactly sorted in the folder and sort order of the links in the sidebar was messed up
3. There were random failures. You could never say when sidebars generation failed (silently), and for the life of me, I could not figure out why. With dozens of files in different folders, I was quite frustrated with all the cut/pasting of content just to see what caused the program to fail
4. If you have a sub-folder with a README in the root, the README does not show in the navigation. This is expected as per Vuepress, but I could never say when the README appeared as a separate link and when it wouldn’t

Solution 2: Use vuepress-bar

vuepress-bar was another ready solution that seemed promising.

First, install the package.

npm i --save vuepress-bar

Next, include the plugin in /docs/.vuepress/config.js -

const getConfig = require("vuepress-bar");

module.exports = {
  themeConfig: {
    ...getConfig(`${__dirName}/..`),
  },
};

That’s it - you should have the new sidebars working. This worked more reliably than solution (1), but I could not figure out how to generate distinct sidebars for indidvidual content folders. All links appeared under one title - which was not ideal for my case.

Consider using this library if you just want a dynamic sidebar (and/or a navbar).

Solution 3: Go custom

My final solution was just doing everything myself. This was not as scary as it initially appeared - all it took was a couple of lines of code.

Get started by doing changes in /docs/.vuepress/config.js -

const fs = require("fs");
const path = require("path");

module.exports = {
  //...
  themeConfig: {
    sidebar: {
      "/why-learn-salesforce/": getSideBar(
        "why-learn-salesforce",
        "Why Salesforce?"
      ),
      "/admin-guide/": getSideBar("admin-guide", "Admin Certification Guide"),
      "/pd1-guide/": getSideBar("pd1-guide", "PD1 Certification Guide"),
      "/salesforce-for-students/": getSideBar(
        "salesforce-for-students",
        "Salesforce for Students"
      ),
    },
  },
  //...
};

function getSideBar(folder, title) {
  const extension = [".md"];

  const files = fs
    .readdirSync(path.join(`${__dirname}/../${folder}`))
    .filter(
      (item) =>
        item.toLowerCase() != "readme.md" &&
        fs.statSync(path.join(`${__dirname}/../${folder}`, item)).isFile() &&
        extension.includes(path.extname(item))
    );

  return [{ title: title, children: ["", ...files] }];
}

And voila, you have the brand new sidebar -

You can see that -

1. There’s a distinct sidebar for each folder - first level links are the files within the folder
2. Links to README and other files are consistent

Here’s a distilled-down version of my experience with Vuepress.

The Situation

I create static sites that are “real” websites. Typically the following factors in a static site generator help -

1. A cool, easy-to-customise home page
2. Themes/starter projects that make life easy
3. Plugins that can extend functionality of core software

Although I had played around Vuepress quite a bit, I always ended up choosing non-Vue technologies like Hugo, or Vue-based static site generators like Gridsome or Saber.land.

More recently however, I decided to have this long drawn out project to create a free tutorial website. With great delight and big plans, I chose Vuepress.

This is that true story.

What exactly is Vuepress?

A Vue-based static site generator, which is created by Evan Yu and supported by the Vue team. If that is not enough to get you going, there are other factors too. Read on.

Vuepress was created for documentation sites but has official themes/plugins to create a blog as well.

• Create documentation sites
• Create full-scale websites (we will get back to this point)
• Create blogs
• Create eCommerce sites
• Make sites interactive with the power of Vue

Vuepress is simple, provides a standard experience, (kind of) easy to configure and easy to get started.

Getting Started on Vuepress

Create a project folder and install Vuepress.

mkdir awedocs
cd awedocs
yarn init
yarn add -D vuepress

And then.. be prepared to act surprised - because there is nothing really there. There is little to see fresh out of the box. We need a few more steps to get there.

Open the awedocs folder in your VSCode. Create a sub directory called docs in the root folder. Then, create a file named README.md in docs folder. Enter following content in the file..

Hello world

md or markdown files will contain the content of your site. This file should be good for initial tests.

Open package.json and enter scripts to run Vuepress. Your entire file may look like below -

{
  "name": "awedocs",
  "version": "0.0.1",
  "description": "Awesome docs.",
  "scripts": {
    "dev": "vuepress dev docs",
    "build": "vuepress build docs"
  },
  "main": "index.js",
  "repository": "https://github.com/techformist/awedocs",
  "author": "Prashanth Krishnamurthy",
  "license": "MIT",
  "devDependencies": {
    "vuepress": "^1.2"
  },
  "dependencies": {}
}

Go back to command prompt, and issue the command ..

yarn dev

.. and ta da - you have your Vuepress site up and running.

You can make a few smart observations -

1. Vuepress is minimal (at least in the beginning)
2. Vuepress comes with a default theme
3. Content and presentation layer is well isolated with the presentation layer

This means that -

1. you can get started really really quickly. Install Vuepress -> start creating documents / blog posts
2. you don’t need to learn Vue/Vuepress to use Vuepress to create a documentation site or blog

Making Vuepress your own

The site generated so far is functional but too minimal. Let’s add some jazz.

Update README.md with following content -

---
home: true
heroImage: /images/dog.jpg
heroText: Awedocs
tagline: docs. docs. docs.
actionText: Get Started →
actionLink: /more
features:
  - title: Good
    details: Know everything.
  - title: Better
    details: Know something.
  - title: Best
    details: Know nothing.

footer: MIT Licensed
---

# Hello world!

The content between --- is called frontmatter. This tells Vuepress to mete out some special treatment to the file.

Now do two things to support above content -

1. Add a file called more.md alongside README.md.

   # Morrreee
   
   This is much more information than I can handle.
   

2. Create a new folder /docs/.vuepress/images. Save a sample image and name it dog.jpg.

Stop Vuepress dev server. Start it again.

yarn dev

You have something that resembles a typical website.

The /docs/.vuepress folder will have any and all customisations for your Vuepress instance. This includes -

• Vue Components
• Navigation bars
• Additional Javascript
• Styles

Content stays in the /docs folder.

Add more UI elements

Lets add a sidebar and navbar.

Create a file called /docs/.vuepress/config.js, which will house all the configuration for Vuepress. Introduce following content in the file -

module.exports = {
  base: "/",

  title: "awedocs",
  description: "Awesome docs",

  themeConfig: {
    label: "English",
    editLinkText: "Edit this page on GitHub",
    lastUpdated: "Last Updated",
    nav: require("./nav"),
    sidebar: [
      { title: "Home", children: [""] },
      {
        title: "Misc",
        children: ["more"],
      },
    ],
    repo: "techformist/awedocs",
    editLinks: true,
  },

  plugins: {
    "@vuepress/back-to-top": {},
  },
};

Create a file called .vuepress/nav.js.

module.exports = [
  {
    text: "Let Me Google",
    link: "https://google.com",
  },
  {
    text: "About",
    items: [
      { text: "About", link: "/about" },
      { text: "Terms", link: "/terms" },
    ],
  },
];

Restart Vuepress to see this -

Build and Deploy

Once you are ready, you do a simple -

yarn build

This will create a docs/.vuepress/dist folder. Upload this folder to your host and you are done!

What could I do next?

Well, sky’s the limit. Vuepress is based off Vue and Javascript - so you can do nothing short of magic.

There are a few things that can be highlighted here -

1. Add simple Vue components and interactivity

We can include interactive components in markdown.

Create a Vue component called AddNum.vue in docs/.vuepress/components/.

<template>
  <div>
    <div>
      <label for="a">Num 1</label>
      <input id="a" v-model="a" type="number" />
    </div>
    <div>
      <label for="b">Num 2</label>
      <input id="a" type="number" v-model="b" align="right" />
    </div>

    <div style="font-weight:bold; margin-top:15px">
      <div>
        <label for="sum">Sum =</label>
        <input id="sum" type="number" :value="sum" align="right" />
      </div>
    </div>
  </div>
</template>

<script>
  export default {
    data() {
      return {
        a: 0,
        b: 0,
      };
    },
    computed: {
      sum() {
        return +this.a + +this.b;
      },
    },
  };
</script>

<style lang="css">
  input {
    text-align: center;
  }
</style>