Developer Discussion Area

[huss]

Please use this area to address topics of interest to developers including questions, changes that impact others, status, etc.

[huss]

There are a number of pending PRs where most are slated for version 0.7. They will mostly be merged in the order they were received where some are waiting on updates. The goal is to release 0.7 in Feb. 2021. Other PRs will be merged after that for version 0.8. Note that several PRs include changes to packages and the database. This will require redoing your node modules along with either migrating or reinstalling your database.

[huss]

I'm starting the merging of the 0.7 PRs and bypassing maps since they are not yet ready. Some of these changes will impact the database configuration and the needed node modules. I'm going to try to come up with a system that tells how each PR impacts these but be careful in pulling from development and then restarting OED. You will need to redo the database and/or node modules as the changes are happening. I'm hoping they all happen in the coming days and I'll try to alert everyone when they are mostly done. You can delay doing an update if you want to avoid constant changes.

[huss]

I've queued this up for the next changes to the website but wanted to point out for developers what will be added:

Other developers and reviewers like to know if this pull request requires any changes in the OED environment. To let others know, please put the following in your pull request description as is appropriate:

• "Note merging this changes the node modules." This indicates you changed package.json so that new packages or versions are needed. This indicates to others that is it not safe to use the "--keep_node_modules" option when installing OED.
• "Note merging this changes the database configuration." This indicates that the database needs to be updated for these changes to work correctly. Normally there will be migration files to update the database. However, many developers will choose to reinitialize the database to do a clean install to be sure everything works correctly.

The hope is this will help you know when to take action.

[huss]

As part of an effort to be more inclusive, I was planning on renaming the branch master to main per this page. I want to give developers a chance to comment on this before I do it. It will happen after a few days unless there are concerns.

[limh0228]

I think this sounds like a great idea!

[huss]

I have now renamed master to main in the OED GitHub repos. This probably will not have much of an effect on you since developers work off of development for the code base. However, you may want to clean this up now in case you need it in the future.

Note that I think you need to first go to your fork on GitHub. (I’m not 100% on this as I could only do it for the web repo as I cannot fork the code repo since I am the admin.) A message should pop up that master was renamed on origin. It has a link to rename the branch on your fork. Click it and then edit the name master to main. If you go back to the main code page for the repo you will probably see this message (same as one I was told to tell everyone about):

---

The default branch has been renamed!

master is now named main

If you have a local clone, you can update it by running:
git branch -m master main
git fetch origin
git branch -u origin/main main

---

Note I had to switch to branch master on origin on my local machine to do these commands in a terminal on my local machine.

This did switch master to main but in VSC there was still an origin/master branch (along with main) that I assume will go away after a while. GitHub has an info page on this at https://github.com/github/renaming.

Note I did the same change on the webpage so you need to do this if you happen to have that cloned on your machine. (And harf too but I doubt anyone has a clone of that.)

[huss]

I'm just venting here. After the change to main branch, both my forked test and the main OED website stopped working. After losing time I found a post that helped. I pushed an empty commit to my forked copy and it reappeared. Apparently it needed that to rebuild and appear. Then I did a PR to do the same to the main OED website. It said it rebuilt but the site with all fine but the website was still dead. I edited one HTML file to add a space (does not change what appears), committed it and then the site appeared. I wish this advise was on the GitHub page about changing master and it would be nice if a fork and the main site acted the same. If anyone knows why this happened they can let me know.

[huss]

As part of my work to get OED on a more solid and open footing as the project grows, I have placed a code of conduct on the project and added a statement to the README file. If you have any thoughts or concerns then please let me know (or post them as a reply).

[huss]

The entire website has been redone and is now live. It has some updates to developer pages but a complete redo of the website and all new help pages. Feel free to use them and look them over to make suggestions.

[huss]

Originally the next release of OED was going to be 0.6. I switched to 0.7 because many new features became part of this release. Now I have decided it is best to go back and use 0.6. I expect we will have several decent size releases before 1.0 so I wanted to leave more numbers between. The next release (was 0.8) will now become 0.7. I just wanted everyone to know so there is not confusion about what each release number means.

[huss]

I am thrilled to announce that OED version 0.6.0 is in the final testing stage. All issues and PRs for this release are now merged into development. I invite all developers to download the latest development and test it out. We have a few days to see if anyone finds any issues that went unnoticed. Note that the changes require a new DB and node_modules (see earlier discussion item) so make sure to do a full install after you get this new version.

I cannot thank all the developers enough who contributed to making this release a reality.

I'm going to try to freeze development except for urgent fixes for the new version. Once it is tagged and official, the PRs in the queue will begin to be addressed.

[huss]

I have now done a reasonably careful check of a running version of OED with the proposed 0.6.0 version. I found some issues now posted on GitHub. However, none are serious enough to delay the release. I'd like this done so OED can move on and unlock development branch changes. Thus, if anyone has any issues with this version please let me know by noon (CST) on Tuesday, March 2. If I don't hear from anyone then I plan to make this the official release after that time.

[huss]

OED v0.6.0 is now live. I will celebrate for a few minutes before I start on the PRs already available for 0.7.0 :-)

[huss]

A couple of changes added to development that developers might want to know about:

1. The dev details page explains how to run one or a few tests using a new command in package.json.
2. I added issue and PR templates to the project. The PR page was updated to reflect this.

[huss]

I've added a few more items to developer web page that might be of interest:

• Dev details describes the 15000 timeout and fix during testing and how to access the database outside OED.
• A new page describes how to load a test map.
• PR page clarified how to see status and details of automated testing.
• The WSL install page (off of getting started) was updated. Esp. of note is the seeming need to install 2 times when you do a new npm install.

Please let me know as you have issues so the pages can continue to be improved.

[huss]

A merge into development that should not have caused an issue led to a GitHub Action test failure (see https://github.com/OpenEnergyDashboard/OED/runs/2122673442). The error was:

1. "before each" hook for "array of reading & moment to array of Reading values":
   error: deadlock detected

I made a simple change to development (unrelated but allowed a push to development) and the issue went away. I also did not see it on my local machine. I'm not sure why it happened but wanted to let others know and to document in case it turns up again.

[huss]

I merged in code to allow for creating of test data without needing to download any files. It also allows better testing of OED. See the developer page about test data that now contains that info. Let me know if you have any issues using the new data.

[huss]

I just approved a PR to slightly increase the version of y18n. I don't think it is critical to do a full install of OED (without keeping the node modules) if you pull this down but I have not carefully tested. I wanted to give everyone a heads up.

[huss]

I've created a private repo named DevDocs that will hold all historical, current and future documentation for development changes. All developers on OED have been granted write access (please edit carefully :-). There is a README.md file to tell what is located in the repo. Files are in markdown so editing on either GitHub or in an IDE should be easy. As I move documents from Google Docs to the new location, I am placing the statement "This has been moved to GitHub DevDocs so that version should now be used." so you know not to use the Google Drive version. At the same time I am taking away the edit ability on the Google Docs. It is possible you have not used the documents yet and that is fine. You only need to look if you need a design document or similar info. Please let me know if you have any thoughts, issues or ideas.

[huss]

PR #621 is causing one of the CodeQL tests to fail for rate limiting (5x). This has been a known issue (see #266). CodeQL has suggested fixes that are not too hard so it is hoped to fix this soon. I wanted everyone to know as this failure is going to show up on all PRs until the fix is done.

[huss]

I wanted to let everyone know that PR #621 changed the DB. If you don't have DB data that your care too much about you can remove postgres-data/ and then install. You can upgrade the DB but it takes a few steps (let me know if you need to do that).

[huss]

Everyone should know that I just merged PR #632. This makes substantial changes to the database so you will need to reinitialize it if you pull this code. I believe the testData option is fixed so it will reload the standard test data. (I am having an issue but I believe it is related to the new version of docker that is experimental and hopefully will not impact others. Let me know if you have any issues.)

[huss]

The new pipeline work was just merged into development (PR #629). This changes the database so you need to upgrade or reinstall DB when this is pulled. Also, the packages were modified so you should do a full install (I have not rechecked but I don't think it was only removal so a full install is needed).

This also changes how test data is loaded into OED so it is much easier. The new info on this should be live on the website soon (let me know if you don't see it when needed).

[huss]

When PR #651 is merged, it will change the DB configuration. There is no migration since this is all internal to release v0.7.0. If you have meters in a development DB that you want to migrate, you should be able to use (within Postgres shell):
ALTER TABLE meters ALTER COLUMN reading SET DATA TYPE FLOAT;
Note that new installs will be fine. Let me know if you have any questions or issues.

[huss]

Note PR #653 that is about to be merged into development updates the node modules. If you pull this into an existing development environment then you need to reinstall the node modules.

[huss]

Development now has all the items we plan for v0.7.0. I have reasonably tested it and not found issues yet. Others are welcome to try it before the formal release planned for Sept. 15. We are moving forward!

[huss]

I am excited that a lot of great work by developers is now in the just released v0.7.0 of OED. We continue our progress toward v1.0 that will allow arbitrary resource types. Many thanks to everyone who helped make this happen.

[huss]

Two recent PR merges changed the database and node packages. PR #679 changed the database and there is a migration for this change (or you can wipe your database and reinstall). PR #677 updated webpack and a number of packages. You need to reinstall the node packages for that to work (docker compose up). The developer details gives some information on doing these types of changes. You will need to deal with this if you pull from development. Please let me know if you have any questions.

[huss]

Noting that PRs #690 and #691 that were merged today change the packages for OED. When pulled you have to update your node packages. We are getting close to release 0.8.0.

[huss]

Development should now be ready for release 0.8.0. I have tested migrations and looked at a few pages. I encourage any developer that can to pull this version (probably requires to either migrate or redo database and possibly node_modules) to try it out. We will have a few days to carefully check for any issues. The current date to set this release is Sunday, January 23 so getting an found issues by Saturday would be appreciated. Hopefully this works as expected. Thanks to everyone who helped make this possible.

I still have a few last minute changes to the help files that should be done soon.

[huss]

I'm very pleased to announce that v0.8.0 has been released. I want to thank the many developers who contributed to making this a reality. This prepared us to move forward toward v1.0 that will allows most types of resources. That work is beginning with the plan to complete that release sometime in the second half of 2022.

[huss]

The plans for resource generalization indicate that there will likely be committed work that will temporarily break parts of OED. This should not be an issue for those working on resource generalization but it could be for other developers. There are also likely to be multiple database changes during resource generalization that other developers can ignore for a while. To deal with this, developers doing different work will work from different branches:

• If you are working on resource generalization then you should work from the resourceGeneralization branch. You can branch from that and do pull requests back to that branch.
• All other developers will work from the development branch as is currently the case.

I expect there will be times when development is merged into resourceGeneralization to check compatibility. We will merge the other way at some point(s) but they will clearly be declared to all developers.

Please let me know if you have any thoughts or questions. Overall, I'm excited that we are beginning this process and have v1.0 within sight.

[huss]

I just merged PR #696 from @jina2k that is another step toward updating all the OED packages. If you pull from the development branch you will need to do a full npm update for this to work. Items in the database should be fine.

[huss]

PR #718 slightly changes the node module dependencies so it would be good to do a full install of them when you merge this PR.