Merge dev into main #29

evalott100 · 2023-09-01T10:31:22Z

dev now contains commits from many different PRs, this PR replaces this one.

The PRs that have already merged into dev are:

If we merge this PR I don't mind closing these open PRs and making new ones into main, alternatively we can merge them first.

Note: dist is also failing on main, once this is merged to main we can make a new issue for this before release.

This does now mean we do a lot more re-ordering of the list, but the alternative is to keep two lists which would more easily become out of sync.

This applies to any PandA field that returns us a label field. Tests updated to accomodate this.

Also fixes some mypy errors in tests

Also upgrade to newest pythonSoftIoc which includes waveform of strings support. Also adjust typing of some test fixtures to better indicate that the order of items is important

Also raise exception if initial connect fails.

Also improves the IOC startup, by waiting for a signal rather than just a flat sleep timer

Changed test to use lower case names consistent with PVA

…d the name of DummyServer to MockedServer

…e working callstack

… retrieving commands from the to give them time

…nd 3.10/3.11. We should consider 3.10 minimum on pandablocks-ioc

Add pcap arm pvi entry

Pvi entries for hdf writing

dmgav · 2023-09-04T19:54:30Z

Would you consider merging #27 before merging dev into main? The PR seems to be ready. The issue is rather critical for testing Panda with fly scans. The changes were discussed with @coretl on Slack.

…ready

AlexanderWells-diamond

See comment regarding listing files

src/pandablocks_ioc/_pvi.py

AlexanderWells-diamond

Looks good! We can finally start having much simpler PRs :)

dmgav · 2023-09-05T13:47:24Z

@evalott100 I see that you did merge #27, but for some reason I don't see the changes from this PR in the main branch. Should I reopen the PR or create a new PR (from the same PR branch) to the main branch?

evalott100 · 2023-09-05T13:52:59Z

@dmgav Yes please do, I'm unsure why those changes aren't there. Sorry for the bother!

I'm happy to approve either a rebase of the original, or a new PR.

dmgav · 2023-09-05T13:55:18Z

@evalott100 Another question: currently the IOC is checking if the directory with .bob files is empty and fails to start if it is not empty, which means that the IOC can not be simply restarted without clearing the directory with bob files. There used to be an option --overwrite-bobfiles, but now it is removed. Are there any plans to implement any option that allow restarting the IOC without removing the bob file (e.g. an option not to generate the bob files if automatic overwrite is not desirable)?

coretl · 2023-09-06T14:43:42Z

I propose changing the positional commandline screen_dir argument to an optional --screen-dir=. If provided it must be an empty directory that will be filled with bob files. If not provided then no bob file generation will take place.

@dmgav, @evalott100 what do you reckon?

AlexanderWells-diamond · 2023-09-06T14:54:35Z

The issue with that is you still have to manually clear the directory between runs, if you do want to generate bobfiles. e.g. you run it once, then make a change to the PandA, then want to regenerate your bobfiles. With your suggestion there's an extra step required before the re-run to delete the files.

I don't know how common this workflow is though, so may be overstating the problem.

dmgav · 2023-09-06T14:56:03Z

I agree with making --screen-dir optional, but I think it would be useful to have another argument that tells IOC to skip generating bob files if the directory is not empty.

coretl · 2023-09-06T15:01:09Z

@dmgav what's the use case for this? I'm interested to see how you intend to run this in production. For us we would probably put the creation of the bob files in a temporary directory, then rsyncing to a shared filesystem in the container startup. Where do you plan to put bob files in production?

coretl · 2023-09-06T15:02:39Z

Also, does generating bobfiles take a long time?

dmgav · 2023-09-06T16:02:57Z

No, execution time is not an issue. Bobfiles are generated quickly. We have not discussed how the IOC is going to be deployed in production. I am currently thinking about the development environment. In principle, making the --screen-dir optional covers all use cases, since it allows to start the IOC without clearing the directory with bobfiles or creating an empty temporary directory. Additional argument could make life easier, since it would allow to start the IOC using fixed set of arguments (e.g. without changing a script that starts the IOC) and simply clear the directory with bobfiles when they need to be generated. This could probably be done by writing a little bit more complicated script though.

evalott100 · 2023-09-07T07:52:17Z

I propose changing the positional commandline screen_dir argument to an optional --screen-dir=. If provided it must be an empty directory that will be filled with bob files. If not provided then no bob file generation will take place.

@dmgav, @evalott100 what do you reckon?

I like this idea!

evalott100 · 2023-09-07T07:59:13Z

No, execution time is not an issue. Bobfiles are generated quickly. We have not discussed how the IOC is going to be deployed in production. I am currently thinking about the development environment. In principle, making the --screen-dir optional covers all use cases, since it allows to start the IOC without clearing the directory with bobfiles or creating an empty temporary directory. Additional argument could make life easier, since it would allow to start the IOC using fixed set of arguments (e.g. without changing a script that starts the IOC) and simply clear the directory with bobfiles when they need to be generated. This could probably be done by writing a little bit more complicated script though.

I can see this use case. I'll add that along with ---screen-dir in a branch.

AlexanderWells-diamond and others added 30 commits March 30, 2023 09:45

Ensure all INP links are valid

337812b

Make the LABELS record the same order as PandA sent

1e25c54

This does now mean we do a lot more re-ordering of the list, but the alternative is to keep two lists which would more easily become out of sync.

Publish enums as lists of strings, not ints

cb19511

This applies to any PandA field that returns us a label field. Tests updated to accomodate this.

Add metadata to PVAccess for last column in table

085453c

Also fixes some mypy errors in tests

Add test for PVAccess

0772b53

Also upgrade to newest pythonSoftIoc which includes waveform of strings support. Also adjust typing of some test fixtures to better indicate that the order of items is important

Set record alarm when PandA disconnects

db80d54

Also raise exception if initial connect fails.

Remove records that we no longer want to publisise

e38323f

Add misc comments

b342348

remove bobfile server

966274f

Refactor how bob file screen dir is specified

4606e56

Fix bobfile tests by regenerating all the files

b4031fc

Also improves the IOC startup, by waiting for a signal rather than just a flat sleep timer

Ignore numpy runtime mismatch error

d3a4f19

Add extra ignore for Python3.9+

3deab0e

Check we create expected number of files

c7499b9

Update src/pandablocks_ioc/_tables.py

a8586b9

Changed test to use lower case names consistent with PVA

ec9cc23

Merge pull request #22 from evalott100/fix_failing_unit_test

61ff502

Changed test to use lower case names consistent with PVA

Changed create_softioc to take a client instead of a hostname, change…

35dd00d

…d the name of DummyServer to MockedServer

Improved functionality of the mocked panda

d14baff

Changed tests to fix the new structure

4b3ebef

made flake8 compatible

e1c4c2d

Changing test back a little to see if I can find the issue with one test

1550517

fixed a unit test

0da76d9

Fixed linting

045c2e5

Still can't get this test working, going to switch branches to see th…

b851fc7

…e working callstack

Using a mocked Asyncio Client for the hdf5 tests

b370cf3

Corrected some parts mentioned in the PR

f055f82

Corrected assorted problems mentioned in the PR

10f70ca

Corrected the time update tests drvl, added sleep statements to tests…

70d15c6

… retrieving commands from the to give them time

Pinned pvi requirement to allow for bobfile consistency between 3.9 a…

51bab11

…nd 3.10/3.11. We should consider 3.10 minimum on pandablocks-ioc

evalott100 added 3 commits September 4, 2023 15:17

Merge pull request #30 from evalott100/add_pcap_arm_pvi_entry

defc6f6

Add pcap arm pvi entry

Merge branch 'dev' into pvi_entries_for_hdf_writing

4606954

Merge pull request #31 from evalott100/pvi_entries_for_hdf_writing

ad1809a

Pvi entries for hdf writing

evalott100 added 3 commits September 5, 2023 09:15

Added a check that the screens directory doesn't contain any files al…

14b6e0c

…ready

Removed filter warnings and flake8 issues

3c75063

Cleaned a regex expression for extracting metadata labels

4b34925

evalott100 mentioned this pull request Sep 5, 2023

Fix dist #33

Closed

evalott100 requested a review from AlexanderWells-diamond September 5, 2023 08:41

AlexanderWells-diamond requested changes Sep 5, 2023

View reviewed changes

src/pandablocks_ioc/_pvi.py Outdated Show resolved Hide resolved

Changed the FileExistsError to not specify files

9fb2d97

evalott100 force-pushed the dev branch from 85b7c0f to 9fb2d97 Compare September 5, 2023 08:54

AlexanderWells-diamond self-requested a review September 5, 2023 08:55

AlexanderWells-diamond approved these changes Sep 5, 2023

View reviewed changes

evalott100 merged commit 6f7232d into main Sep 5, 2023
15 of 16 checks passed

evalott100 deleted the dev branch September 5, 2023 09:00

This was referenced Sep 7, 2023

Add new command line args #41

Closed

Added new command line arguments #42

Merged

evalott100 self-assigned this Oct 11, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Merge dev into main #29

Merge dev into main #29

evalott100 commented Sep 1, 2023 •

edited

Loading

dmgav commented Sep 4, 2023

AlexanderWells-diamond left a comment

AlexanderWells-diamond left a comment

dmgav commented Sep 5, 2023

evalott100 commented Sep 5, 2023

dmgav commented Sep 5, 2023

coretl commented Sep 6, 2023

AlexanderWells-diamond commented Sep 6, 2023

dmgav commented Sep 6, 2023

coretl commented Sep 6, 2023

coretl commented Sep 6, 2023

dmgav commented Sep 6, 2023

evalott100 commented Sep 7, 2023

evalott100 commented Sep 7, 2023

Merge dev into main #29

Merge dev into main #29

Conversation

evalott100 commented Sep 1, 2023 • edited Loading

dmgav commented Sep 4, 2023

AlexanderWells-diamond left a comment

Choose a reason for hiding this comment

AlexanderWells-diamond left a comment

Choose a reason for hiding this comment

dmgav commented Sep 5, 2023

evalott100 commented Sep 5, 2023

dmgav commented Sep 5, 2023

coretl commented Sep 6, 2023

AlexanderWells-diamond commented Sep 6, 2023

dmgav commented Sep 6, 2023

coretl commented Sep 6, 2023

coretl commented Sep 6, 2023

dmgav commented Sep 6, 2023

evalott100 commented Sep 7, 2023

evalott100 commented Sep 7, 2023

evalott100 commented Sep 1, 2023 •

edited

Loading