Using git

As a developer I am collaborating on multiple projects. In many cases it is need to create a fork of the upstream project add add a pull request. The first steps (making a fork) etc are not difficult, and at first you will have an up to date clone of the original website on which you can work and open pull requests to the upstream project. Most of these steps can be down using the de web UI (e.g. when using Github). But while working on a pull request often it is needed to sync your forked clone, for instance to open a new branch. Then the trouble begins. What can happen:

  • You started a new branch , but the main branch was not in sync
  • You need to rebase and possible there is a conflict

People start rebasing but often pull in unrelated commits causing other issues. In this post I want to share some of methods I have to deal with such cases.

Opening a fresh branch

When you want to open a new pull request you want this to happen correctly. I assume here that you already have set up some develop environment with your cloned git repository, ready to push commits.

To make sure your fork is in sync you need to fetch all upstream updates. We can do that with:

git fetch --all

or if it is just upstream, we can use

git fetch upstream

which is faster.

The next step is to checkout the target branch you want to create your pull request against. This can be main, master or dev, but sometimes this can be even different. In our example we use main.

git checkout main 

Now we want to reset main to the same state as upstream\main, as this is the branch where we want to target the pull request to. We need to make sure there are no uncommitted files as these will be removed!

git reset --hard upstream/main

If we omit the --hard flag then git will create uncommitted files for all changed between the old and new state. This can be handy if we want to revert commits but not when starting a new branch.

From here we can push the local branch to our fork with:

git push

Now we are at the same level as the target branch and we can open a new branch that is in sync with upstream. We choose a new branch name that reflects the name of our pull request, in this example I’ll use test-pull-request as branch name.

git checkout -b test-pull-request

This creates a new local branch, from here you can start coding and add your commits. If you are ready you can publish your branch to your fork (I use origin as name for the cloned repository). From then every new commit will be pushed to origin as well. When we publish our new local branch, then we should set the upstream branch too so we can open a pull request later.

git push --set-upstream origin test-pull-request

When all commits are done and we pushed all our commits (git push), then we are ready to open a pull request.

Open a new pull request

When using Github it is advisable to open a pull request using the web UI or using a plugin in vscode or other development environment. In most cases there is a template that needs to be filled in. Make sure the correct commits are shown and you have selected the correct target branch!

Rebasing and resolving merging issues

Sometimes we need to rebase. This can happen if the PR is to old and we need to sync with our target branch. Personally I would like to force push the original commits on top of the rebased branch instead of adding a merge commit. Force pushing makes it easier to keep track on the commits in the pull request and avoids additional merging commits. To rebase correctly first we need to fetch the latest updates. We can use:

git fetch upstream

Now we make sure we are checked out to the correct branch.

git checkout test-pull-request

To rebase with our upstream target branch we start a rebase command.

git rebase upstream/dev

If you want to select the commits that should be included (for instance when you want revert some commits) you should consider using the -i flag to start interactive rebasing.

git rebase -i upstream/dev

For each commit include git will try to rebase, if this fails, then you need to correct, save and stage your files to tell git which changes should be made. After staging you can continue rebasing:

git rebase --continue

Make sure you only make changes to commits that are directly related to the merging conflicts and reflect the wanted changes for that particular commit. Repeat this for the other commits (if any) till the rebase is finished successfully.

If all becomes a mess, then there is a bail out of this by aborting the merge operation.

git rebase --abort

Next step is that we no NOT just pull and push commits now but only force push the rebased commits. This can illogical but what we want is to make sure that we push the exact local situation to our origin branch.

git push --force

Now our branch should be rebased and in the pull request we should only see the commits of our PR, our the selected commits from the file (when using the -i option).

Creating new clean commits from previous work

If your have a larger PR with multiple files and you would like to replace or revert existing commits without loosing your work you can use git reset.

With git log we can show the commits we have made on top of our branch. To replace these commits we need to rebase to the commit from where we want to start again.

Say we have two commits we want to do over. Show them with git log.

commit 6090d321e3926ad9c5ffdd026f7c2fb046cdbbf2 (HEAD -> test2)
Author: you <you@example.com>
Date:   Wed May 24 14:19:24 2023 +0000

    commit 2

commit a0ad22921fb8f5797aeb4e414cf3403b80027a3d
Author: you <you@example.com>
Date:   Wed May 24 14:18:03 2023 +0000

    commit 1

commit abf08f66a4c7e01955213c228542884951d45a11 (upstream/dev, origin/test2, origin/dev, origin/HEAD, dev)
Author: user <user@users.noreply.github.com>
Date:   Wed May 24 01:38:16 2023 -0500

    Some commit upstream

Assuming we have no uncommitted files we can rebase to commit abf08f66a4c7e01955213c228542884951d45a11 (upstream/dev, origin/test2, origin/dev, origin/HEAD, dev).

git reset abf08f66a4c7e01955213c228542884951d45a11

Now you will see that the changes of the last 2 commits are unstaged files now. You can now stage the changes for your new commit, or if you want do this in more commits until all changes are staged and committed. Now, to replace the old commits with the new ones we can force push them to our origin branch.

git push --force

Now your commits have been replaced with the new ones.

Temporary save uncommitted work

Sometimes you might need to change between branches, but you have uncommitted files. If pre-commit is used, it might not be possible to commit. Instead you could stash your changes on the stack. You can get back your changes later by popping them from the stack. This also works if you want to apply uncommitted changes to a different branch.

To stash your uncommitted work:

git stash

To pop back the changes:

git stash pop

There is a lot more you can do with git stash, but I will keep it brief here.

For more git commands you can use the online documentation.

Elro Connects -Home Assistant

I am working on an integration for Elro Connects. The integration should allow users with Elro Connects fire, water or CO alarms to integrate those in Home Assistant. If you own Elro Connects fire alarms and like to test, you could add the elro connects integration with HACS.

Add the following repo to HACS https://github.com/jbouwh/ha-elro-connects/. You need to restart Home Assistant after downloading the custom integration. After the restart you should be able to add the integration to Home Assistant.

For each alarm a siren entity is created. If you turn it on, a test alarm request will be sent. You can turn it off to silenece a (test) alarm.

Let me know if you like this integration. If your device is not supported, or not working correctly, then let me known!

InfluxDB 2 support

Omnikdatalogger v1.11.1 now has native support for InfluxDB v2 authentication. It is no longer needed to supply v1 authentication support. To enable it you need to configure the config parameters org, bucket, and token. If you use HACS via AppDaemon, make sure you add influxdb-client to the python packages.

Additional you now can use an SSL connection.

Read more about he new release:

https://github.com/jbouwh/omnikdatalogger/releases/tag/v1.11.1-1

Omnik data logger news update

PIP support for omnikdatalogger and omnikdatalogerproxy

Omnikdatalogger can now be installed using PIP:

sudo pip3 install omnikdatalogger

To install only omnikdataloggerproxy use sudo pip3 install omnikdataloggerproxy

HACS default support

Omnikdatalogger could be installed as a custom repository within HACS. A request to include Omnik data logger in the default repository was requested and approved.

Caching of manual inputs for Home Assistant

I found it to be annoying Home Assistant is recovering the manual input selects I am using for my automations. To enable a persistant state I have made a generic scrits that cashes the state of these input to disk and restores them automatically when Home Assistant restarts.

The script is written in Python and is written to work with AppDaemon.

The code shows how to use the Home Assistant integration API that comes with AppDaemon.

New release Omnik datalogger

Check https://GitHub.com/jbouwh/omnikdatalogger for the newrelease (0.91-beta) of Omnik datalogger.

The last days omnikport was down. In the mean while I found a second method getting access your the data. During the outage the portal at https://www.solarmanpv.com/portal was available the whole time. Here you can find your data aswell. This proves omnikportal is just a skin and the data is at solarman PV. I found an API and working code that gives access to the realtime information. This API seems to give more accurate and in time access to the most recent data that was logged. Unfortunally the API is not TLS encrypted and uses a MD5 hashed password login. Not very safe. Neverless the alternative API is a good alternative for the time being. In the main menu there is now a special page for the omnik datalogger sofware.

Pluggable client

The new release comes with a pluggable client for logging. This makes it easy to change the logging code by just changing the configuration.

New client in development for local logging

For older Omnik inverters it is not possible to get the data straight from the inverter. Newer inverters listen at port 8899 and are able to respond directly. The datalogger in the inverter sends an update about every 5 minutes to a fixed IP of a logging server in the Solarman datacenter. The IP address the inverter is logging to is fixed and cannot be changed via the interface, but it is possible to intercept the message by simulating the server using a NAT rule and a static route for the IP address in your netwerk. The code te parse the data that is logged is already available to use, thanks to Woutrrr. The software will be able to listen on a TCP port to the rerouted datalogger sessions. The code ewill be based on this project of t3kpunk. A tutorial how to use this will apear on this site as soon as the code is ready for release. Local logging will give more detail information then currently available using the API’s. Extra sensors will be:

  • AV Voltage* and Current and Frequency (3 fase if yor model supports it)
  • DC Voltages and Currents for all strings (max 2)
  • The Temperature* of the inverter

*These values are submitted to PVoutput as well. And every new sensor will become available via the MQTT plugin.

DSMR integration

For ouput to pvoutput.org I have planned to make an integration for the Dutch Smart Meter (DSMR). An adapter from the P1 connector to a USB serial interface is general available. Integrating makes is possible to calculate the real energy consumption. Of course all these new sensor information will be available to the MQTT output plugin as well.

Outage omnik portal

The last few days, the omnik portal and app and API have been unavailabe. So the data loggging is not possible at the moment. I have found out Omnik dataloggers log to the database of SolarmanPV. It seems you can still see your live output and history.

The URL is https://www.solarmanpv.com/portal/LoginPage.aspx and you can use your omnik portal credentials. I am working to get the API working so the software can be updated. Unfortunally there is only a working unsecure API interface. There is a secure alternative, but an API-key is needed for that. I am working on this.

Have some patience, I will come with a working update coming days.

Regards,

Jan

Thirst (pre) release for the omnik data logger

The first pre-release of the new Omnik data logger is now available. I would like to know if this works for you! If you have any comments or issues, let me know!

Release details

Version: 0.9-alpha (pre-release)

This pre-release brings a solution to integrate the actual state of your Omnik power inverter in to your favorite home automation system. The script reads out omnikportal.com using your credentials and enables you process the real time data of your solar plant.
The realetime output can be forwared to pvoutput.com or send over MQTT to your favorite Home Automation porject. The MQTT implementation supports the Home Assistant Automatic MQTT discovery.

You can run the script from the command line, use systemd or make use of AppDaemon so you can integrate directly with Home Assistent.

Have fun! Let me know your experience if you have any issues.

You can download the release here.