From 2e9c03c95da151f43c936692e2790b86a5437d60 Mon Sep 17 00:00:00 2001 From: Karan Jayachandra Date: Mon, 9 Dec 2024 18:26:01 +0100 Subject: Added a new entry about Python --- content/nano/high_frequency.md | 12 ++++++++++++ content/nano/test.md | 12 ------------ content/nano/the_turn.md | 39 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 51 insertions(+), 12 deletions(-) create mode 100644 content/nano/high_frequency.md delete mode 100644 content/nano/test.md create mode 100644 content/nano/the_turn.md diff --git a/content/nano/high_frequency.md b/content/nano/high_frequency.md new file mode 100644 index 0000000..dc1fb93 --- /dev/null +++ b/content/nano/high_frequency.md @@ -0,0 +1,12 @@ ++++ +title = 'Low quality high frequency' +date = "2024-12-02" ++++ + +# Low quality high frequency + +I have always enjoyed reading. My parents always encouraged my reading habit when I was a child and that stuck with me. With that came an appreciation of good writing. As an child brought up in rural India, I was above average when it came to writing for school. But as I grew up, I realized how woefully incompetent I was at. This was demotivating and resulted in me to stop trying. So for the last 10 years or so, I didn't put pen and paper together other for writing for work or school. All this changed recently. First, I turned 30 recently which was a wake up call of sorts. I suddenly didn't care anymore for if I was incompetent or not. All I wanted to say to myself when I was 40 was that I had given it an honest to god shot. Secondly, every time I saw good writing, I was envious. I was envious of how this author was always going to be better than me at something that I cared about. + +While thinking about this, I realized something. I play badminton and can call myself objectively an advanced player without hubris. The advice I would give to a person that just started without doubt would be to just start and play as much as possible. Ask the people who are better than you for advice and try to one up yourself everyday. Then it hit me, that these authors that were better than me at this skill would tell me the same thing. Come to think of it, this applies to anything that you would like to learn. So I have decided to start writing at least once a week for the rest of the year. The topic can be as vague and the article can be of shoddy quality. But the important thing is to put things out there. + +Therefore, I am going to use this 'nanoblog' of sorts to create posts that are writing exercises of sorts. They are mainly for myself and may not have much value for readers. But I would like to look back on this first article one day and say to myself, "Boy am I glad that I stuck to it and don't write like that anymore." \ No newline at end of file diff --git a/content/nano/test.md b/content/nano/test.md deleted file mode 100644 index dc1fb93..0000000 --- a/content/nano/test.md +++ /dev/null @@ -1,12 +0,0 @@ -+++ -title = 'Low quality high frequency' -date = "2024-12-02" -+++ - -# Low quality high frequency - -I have always enjoyed reading. My parents always encouraged my reading habit when I was a child and that stuck with me. With that came an appreciation of good writing. As an child brought up in rural India, I was above average when it came to writing for school. But as I grew up, I realized how woefully incompetent I was at. This was demotivating and resulted in me to stop trying. So for the last 10 years or so, I didn't put pen and paper together other for writing for work or school. All this changed recently. First, I turned 30 recently which was a wake up call of sorts. I suddenly didn't care anymore for if I was incompetent or not. All I wanted to say to myself when I was 40 was that I had given it an honest to god shot. Secondly, every time I saw good writing, I was envious. I was envious of how this author was always going to be better than me at something that I cared about. - -While thinking about this, I realized something. I play badminton and can call myself objectively an advanced player without hubris. The advice I would give to a person that just started without doubt would be to just start and play as much as possible. Ask the people who are better than you for advice and try to one up yourself everyday. Then it hit me, that these authors that were better than me at this skill would tell me the same thing. Come to think of it, this applies to anything that you would like to learn. So I have decided to start writing at least once a week for the rest of the year. The topic can be as vague and the article can be of shoddy quality. But the important thing is to put things out there. - -Therefore, I am going to use this 'nanoblog' of sorts to create posts that are writing exercises of sorts. They are mainly for myself and may not have much value for readers. But I would like to look back on this first article one day and say to myself, "Boy am I glad that I stuck to it and don't write like that anymore." \ No newline at end of file diff --git a/content/nano/the_turn.md b/content/nano/the_turn.md new file mode 100644 index 0000000..4b7244a --- /dev/null +++ b/content/nano/the_turn.md @@ -0,0 +1,39 @@ ++++ +title = 'Python for Scientific Computing' +date = "2024-12-09" ++++ + +# Python for Scientific Computing +I am a signal processing engineer and most of my work involves the creation of algorithms that make sense of data. I work with the data that is usually generated from an automotive radar but this necessarily doesn't have to be the case. For me, the data could be from any source but as long as there is information in the data, I can find a way to extract it. The complication that arises with my line of work comes from the fact that the data is usually [noisy](https://en.wikipedia.org/wiki/White_noise). Therefore, I end of doing quite a bit of statistical simulations to verify my work. + +Most engineers and scientists in my field work with MATLAB®. I however prefer to use Python. So if you are someone doing something similar to what I have just described, you might find this post useful. The following are some of the learnings from 2-3 years of consistent use of Python code scientific computing. This is of course a subjective account but I am free to discuss any of the points below via [email](mailto:mail@karanjayachandra.com). + +|![Filtering Noise](https://cdn.prod.website-files.com/5babb9f91ab233ff5f53ce10/608a88ee9c90126452af58d8_SMA-filtering.jpeg)| +|:--:| +| *Filtering Noise (Source: [Catchpoint](https://www.catchpoint.com/blog/signal-vs-noise))* | + +## The don'ts +I will start with a few don't that I believe are counter productive if you want to be good at scientific computing. + +### Don't reinvent the wheel +If you are a software engineer, you can ignore this. But if you are trying to do signal processing, do not reinvent the wheel. Do not write your own `fourier_transform()` function. If you are learning to optimize the fourier transform or trying to understand it, by all means, go ahead. But most of the time, stealing someone else's implementation will be much more productive. Most importantly, your implementation will be slower. This will impact your simulation time. Learn to use [numpy](https://numpy.org/doc/2.1/index.html) and [scipy](https://scipy.org). + +### Don't use the latest version of Python +Python is work in progress. Several versions of python exists and I have found that the sweet spot for the version to use is one lower than the latest. Most packages on PyPI will support it and you will not run into issues where a dependency of a dependency of your package is not supported by the latest version. Python is notoriously difficult at handling this. + +### Don't over engineer your code +I have noticed that most of the code that actually helps make scientific conclusions can be achieved via very little. Keep your code and data simple. You don't need inheritance, you don't need that shiny new data structure. Keep to the basics and things will be much more easy to manage in the long run. It is not about how many lines of code but how few lines of code. Reduce the number of moving parts. A side note: If you are working with a team on a large project. Work on a smaller repository for your idea with a simplified test bench to prove your idea before shipping it to the larger system. + +## The do's + +### Use version control +Even if you are the only person working on a project. Use git. This is crucial. If you want to work with other, it is doubly more important. git is not restricted to software projects but any text files that require frequent revision. Even this blog is under version control. Do not have 10 versions of your code floating around. Work on two branches, `main` and `develop` and keep them regularly in sync. + +### Use a virtual environment +As mentioned earlier, python is known to be difficult to handle with respect to dependencies. Dependencies in python are both a boon and a curse. I find PyPI to be a democracy of sorts with the best package rising to the top for use. This allows for programmers to help each other out and reuse things. However, the version of package that you use might not be the same as the one installed on your colleagues machine. Therefore, for every git repository you have on your machine maintain a [venv](https://docs.python.org/3/library/venv.html). Maintain a `requirements.txt` file in the root folder so that others can easily install the dependencies needed for your code. + +### Automate the testing +Most of the time, simulations require the generation of synthetic data or reading stored data and running it through your code. In my experience this is the most time consuming part of scientific computing. Learn to automate this and it will improve your workflow tremendously. Bonus points if you can make this work with git on pushes to the git server via [Jenkins](https://www.jenkins.io) some other automated testing platform. + +### Document the code +At least, maintain a README.md for your repository. Explain briefly what you code does and how to install dependencies and run it. Start with this and when the experiment starts showing results, start maintain a more long form report containing the ins and outs of the code and the mathematics behind it. I prefer [LaTeX](https://www.latex-project.org) for its superior typesetting and ubiquity in academia. It also exports the same report as a HTML document which you can server as a static page along with your code. \ No newline at end of file -- cgit v1.3.1 From fcd789b495073887627b6fb7fc5d4fbe9a853f93 Mon Sep 17 00:00:00 2001 From: Karan Jayachandra Date: Mon, 9 Dec 2024 18:32:25 +0100 Subject: Changed the archetype to automatically add the title --- archetypes/default.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/archetypes/default.md b/archetypes/default.md index c6f3fce..9723798 100644 --- a/archetypes/default.md +++ b/archetypes/default.md @@ -3,3 +3,5 @@ title = '{{ replace .File.ContentBaseName "-" " " | title }}' date = {{ .Date }} draft = true +++ + +## {{ replace .File.ContentBaseName "-" " " | title }} -- cgit v1.3.1