Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP]: expose inputs and port namespacing #1070

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
1af76df
Allow namespacing in inputs and outputs of a ProcessSpec
sphuber Aug 24, 2017
2782b30
Add 'exclude' option to spec.expose_inputs
sphuber Aug 25, 2017
7e98948
Implement the 'include' option for ProcessSpec.expose_inputs
sphuber Aug 25, 2017
44d635c
Add test where the same dict is passed for two namespaces
greschd Aug 30, 2017
4e70099
Fix incorrect test
greschd Aug 30, 2017
252ebb1
Change 'include' default to None
greschd Aug 30, 2017
609e67a
Add simple tests with no and one include
greschd Aug 30, 2017
b3b7eaf
Merge pull request #3 from greschd/fix_660_expose_workchain_inputs
sphuber Aug 31, 2017
2bbbbb4
Merge pull request #2 from greschd/add_test_pass_same_dict_twice
sphuber Aug 31, 2017
25c7a5d
Merge branch 'develop' into fix_660_expose_workchain_inputs
sphuber Aug 31, 2017
cc60f4f
Fix typo in agglomerate tests
greschd Aug 31, 2017
a58625c
Merge pull request #6 from greschd/fix_agglomerate_test
sphuber Aug 31, 2017
9e31ae0
Merge branch 'fix_660_expose_workchain_inputs' of github.com:sphuber/…
sphuber Oct 31, 2017
8e9fd9d
Got rid of a couple of instances of new_runner
muhrin Jan 18, 2018
d0d5197
Made control panel a context manager
muhrin Jan 18, 2018
307bfd9
Centralised the RMQ config
muhrin Jan 18, 2018
915f788
Replace local encode/decode with aiida.utils.serialize
sphuber Jan 18, 2018
40a00b5
Updated to latest plum
muhrin Jan 19, 2018
8a11e70
Merge remote-tracking branch 'muhrin/workflows' into fix_1054_new_dae…
sphuber Jan 19, 2018
f698c6e
Add a logging handler and property dictionary for plum
sphuber Jan 19, 2018
562c144
Fix direct uses of datetime.now for aiida.utils.timezone
sphuber Jan 19, 2018
a6af213
Fixed RMQ stuff to use prefix properly
muhrin Jan 19, 2018
f20e87d
Serialize and deserialize process inputs before passing over rmq
sphuber Jan 19, 2018
2a6a259
Create new verdi entry point to run daemon: verdi devel run_daemon
sphuber Jan 19, 2018
3aa049e
Clean up the get_profile_config function
sphuber Jan 19, 2018
b58c4bc
Merge remote-tracking branch 'muhrin/workflows' into fix_1054_new_dae…
sphuber Jan 19, 2018
fef76da
Merge branch 'fix_1054_new_daemon_console' into fix_660_expose_workch…
sphuber Jan 19, 2018
4699a13
Remove PortNameSpace and exposed_inputs defs from aiida
sphuber Jan 19, 2018
26ce4db
Update travis.yml to point to feature_spec_namespace branch of plum
sphuber Jan 20, 2018
8f00fe4
Merge branch 'workflows' of github.com:muhrin/aiida_core into workflows
sphuber Jan 21, 2018
df429c2
Merge branch 'workflows' into fix_660_expose_workchain_inputs
sphuber Jan 21, 2018
ae95581
Add tests for the get_inputs_template method
sphuber Jan 23, 2018
8341246
Fixed plum change
muhrin Jan 23, 2018
05b6d0f
Merge branch 'workflows' of ssh://github.com/aiidateam/aiida_core int…
muhrin Jan 23, 2018
dac4229
Reverted communicator to blocking
muhrin Jan 23, 2018
603db74
Merge branch 'workflows' of github.com:muhrin/aiida_core into fix_660…
sphuber Jan 24, 2018
73200c5
Fix bug with SinglefileData retrieval in the execmanager
sphuber Jan 24, 2018
fa2b6be
Conform aiida.work to changes in plum related to port namespaces
sphuber Jan 24, 2018
1c77cd1
Fix camelcasing in test file name
sphuber Jan 24, 2018
891a51a
Add test for JobCalculation with a use method with additional parameter
sphuber Jan 24, 2018
bb5e981
Replace InputGroup and input_group with PortNamespace and input_names…
sphuber Jan 24, 2018
d3fd212
Fix test entry point name for work.process_spec
sphuber Jan 24, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ install:
- pip install -U pip wheel setuptools
# Install AiiDA with some optional dependencies
- pip install .[REST,docs,atomic_tools,testing,dev_precommit]
- pip install -e git+https://github.com/muhrin/plumpy@develop#egg=plumpy-0.10.0
- pip install -e git+https://github.com/sphuber/plumpy@feature_spec_namespace#egg=plumpy-0.10.0

env:
## Build matrix to test both backends, and the docs
Expand Down
2 changes: 1 addition & 1 deletion aiida/backends/tests/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@
'work.daemon': ['aiida.backends.tests.work.daemon'],
'work.persistence': ['aiida.backends.tests.work.persistence'],
'work.process': ['aiida.backends.tests.work.process'],
'work.processSpec': ['aiida.backends.tests.work.processSpec'],
'work.process_spec': ['aiida.backends.tests.work.process_spec'],
'work.rmq': ['aiida.backends.tests.work.test_rmq'],
'work.run': ['aiida.backends.tests.work.run'],
'work.runners': ['aiida.backends.tests.work.test_runners'],
Expand Down
84 changes: 81 additions & 3 deletions aiida/backends/tests/work/job_processes.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,16 +10,49 @@

from aiida import work
from aiida.backends.testbase import AiidaTestCase
from aiida.common.utils import classproperty
from aiida.orm.data.base import Int
from aiida.orm.calculation.job.simpleplugins.templatereplacer import TemplatereplacerCalculation
from aiida.work.class_loader import ClassLoader
from aiida.work.job_processes import JobProcess

from . import utils

Job = TemplatereplacerCalculation.process()
class AdditionalParameterCalculation(TemplatereplacerCalculation):
"""
Subclass of TemplatereplacerCalculation that also defines a use method
with an additional parameter
"""

@classproperty
def _use_methods(cls):
retdict = TemplatereplacerCalculation._use_methods
retdict.update({
'pseudo': {
'valid_types': Int,
'additional_parameter': "kind",
'linkname': cls._get_linkname_pseudo,
'docstring': (''),
},
})
return retdict

@classmethod
def _get_linkname_pseudo(cls, kind):
"""
Create the linkname based on the additional parameter
"""
if isinstance(kind, (tuple, list)):
suffix_string = '_'.join(kind)
elif isinstance(kind, basestring):
suffix_string = kind
else:
raise TypeError('invalid additional parameter type')

return '{}_{}'.format('pseudo', suffix_string)

class TestJobProcess(AiidaTestCase):

def setUp(self):
super(TestJobProcess, self).setUp()
self.assertEquals(len(work.ProcessStack.stack()), 0)
Expand Down Expand Up @@ -54,7 +87,8 @@ def test_job_process_set_label_and_description(self):
'_label': label,
'_description': description
}
job = Job(inputs)
process = TemplatereplacerCalculation.process()
job = process(inputs)

self.assertEquals(job.calc.label, label)
self.assertEquals(job.calc.description, description)
Expand All @@ -76,4 +110,48 @@ def test_job_process_set_none(self):
'_description': None
}

Job(inputs)
process = TemplatereplacerCalculation.process()
job = process(inputs)

class TestAdditionalParameterJobProcess(AiidaTestCase):

def setUp(self):
super(TestAdditionalParameterJobProcess, self).setUp()
self.assertEquals(len(work.ProcessStack.stack()), 0)
self.runner = utils.create_test_runner()

def tearDown(self):
super(TestAdditionalParameterJobProcess, self).tearDown()
self.assertEquals(len(work.ProcessStack.stack()), 0)
self.runner.close()
self.runner = None
work.set_runner(None)

def test_class_loader(self):
cl = ClassLoader()
AdditionalParameterProcess = JobProcess.build(AdditionalParameterCalculation)

def test_job_process_with_additional_parameter(self):
"""
Verify that the additional parameter use_method 'pseudo' is supported
"""
label = 'test_label'
description = 'test_description'
inputs = {
'_options': {
'computer': self.computer,
'resources': {
'num_machines': 1,
'num_mpiprocs_per_machine': 1
},
'max_wallclock_seconds': 10,
},
'pseudo': {
'a': Int(1),
'b': Int(2),
},
'_label': label,
'_description': description
}
process = AdditionalParameterCalculation.process()
job = process(inputs)
3 changes: 2 additions & 1 deletion aiida/backends/tests/work/persistence.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
import aiida.work.utils as util
from aiida.work.test_utils import DummyProcess
from aiida import work
from aiida.work.launch import run


class TestProcess(AiidaTestCase):
Expand All @@ -34,7 +35,7 @@ def test_save_load(self):
del process

loaded_process = saved_state.unbundle()
result_from_loaded = work.run(loaded_process)
result_from_loaded = run(loaded_process)

self.assertEqual(loaded_process.state, work.ProcessState.FINISHED)

Expand Down
Loading