Skip to main content

Posts

Showing posts from June, 2022

How to make table schema changes and restore data after appropriate transformation

Introduction In many cases we might have to encounter scenarios where we need to perform backup of data or perform a schema change like data type transformation etc. This blog post illustrates the steps that can be used as a checklist to perform the operation at ease Steps The below sql query creates a new table with the same schema as given in the like field create table <table_name>_old like <table_name>; Now that we have the schema ready, we can copy the data for backup using the below command insert into <table_name>_old select * from <table_name>; Once the above command succeeds, we can drop the old table drop table <table_name>; Then we can create the new table with the same name as the one given in <table_name>   so that we can allow applications to still use the same table name. CREATE TABLE `<table_name>` (   `id` bigint NOT NULL AUTO_INCREMENT, ...); Now that the table with new schema is ready, we can migrate the data

using mysqldump command to backup the data from MySQL Database table

Introduction In this blog post, we will take a look at the mysqldump command that is shipped as part of the MySQL clients. Data Backup We normally deal with backup of data by exporting the data as JSON / CSV or as SQL Files. This is required so that any change that we are working on might have a impact, we can immediately use the backup to fix any unexpected database operation. When working with JSON / CSV at times, we have to deal with the issues in the data formats. Further as SQL queries, we get the option to rewrite the query, insert them into a local database, perform the required transform or operations and then move to the server. The MySQL Workbench provides the export option through which we can generate the SQL scripts, but there are issues like the target table names are empty and there is a new insert statement for each row in the table. Example Command mysqldump.exe --host=servername --port= 3306 --default-character-set=utf8 --user=(db_user_name) -p --protocol=tcp --colu

http/3

Introduction HTTP/3 is the 3rd major version of the HTTP protocol that powers the internet. In comparison with the previous version of http which relied on TCP, http/3 relies on QUIC. QUIC Specification published at 6th June, 2022 Developed initially at Google in 2012, had its way to IETF and public by 2022. A decade for completion and standardization !!! Are the semantics like requests, methods, status codes still the same like previous versions of http/2? yes, they still remain the same The underlying mechanisms are changed, like http/3 uses space congestion control over UDP (User Datagram protocol) Head-Of-Line Blocking (HOL) This is a performance problem where there is a queue of packets built due to the first packet that is yet to be consumed. We have browsers that have limits on the number of parallel requests that it can send to a server, when they are used up, as we anticipate, a Queue is formed for the newer requests that start accumulating the newer requests till the former

Split a very large file in to chunks

Introduction Many a times we are having to deal with the very large files, be it the logs from the server like catalina.out    or any other huge sql files or csv or text files. These are some artifacts that could have been generated in the server or might contain the data. There might not have been detailed logging systems that can do a RollingFile like appender that can get the files into smaller chunks for us to use easily. In some cases these could be large data which is hard to split into multiple chunks in an export mechanism. Problem Statement When we try to open them in an editor like  Notepad / Notepad++ / VS Code , we normally run into memory issues and the application gets frozen. At times, in case of windows, we might have to go to  Task Manager  , find the application and then  End Task  so that we can use the other apps or to allow that particular application to be usage Solution In this post, we explore on using git bash a tool. We normally use a git bash command line in

Upgrade from http1.1 to http2 for Java spring boot applications hosted in tomcat

In this post, we will list down the tasks to be done for enabling the HTTP 2.0 support in spring boot applications which are hosted in Apache tomcat webserver Application Level Changes Spring boot Application Configuration Changes server.http2.enabled=true In the spring boot application's application.properties file, we have to add the above line so that Spring boot can add the support for http2 Tomcat server configuration In the tomcat web server, we should have SSL enabled before doing the below change. To start with, we have to shutdown the tomcat server instance that is running CD to the directory that has tomcat installed and cd to the bin directory and run the below command sh shutdown.sh We have add the UpgradeProtocol  which adds the respective Http2Protocol handler classname to the connector pipeline that enables support for http2.0 <UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" /> The above UpgradeProtocol can be added to the connec