Failure to update splunk_saved_search

[igaskin]

Steps to reproduce:

1. Create a splunk_saved_searches resource
2. Edit the resource
3. Apply Changes
4. Apply will fail with a 400 Error

Expected Result:

• making changes to a splunk_saved_searches resource should correctly update the saved search

error output

Warning: Applied changes may be incomplete

The plan was created with the -target option in effect, so some changes
requested in the configuration may have been ignored and the output values may
not be fully updated. Run the following command to verify that no other
changes are pending:
    terraform plan

Note that the -target option is not suitable for routine use, and is provided
only for exceptional situations such as recovering from errors or mistakes, or
when Terraform specifically suggests to use it as part of an error message.


Error: Http Error: [400] 400 Bad Request {"messages":[{"type":"ERROR","text":"Argument \"schedule_priority\" is not supported by this handler."}]}

  on main.tf line 1, in resource "splunk_saved_searches" "default":
   1: resource "splunk_saved_searches" "default" {

[anushjay]

@igaskin - I don't see the same issue when using the example.tf in the repo as an admin user. Does the user have proper permissions to make the schedule_priority change in your case?
The provider basically uses the rest endpoints provided by Splunk and the REST API documents suggests that the edit_search_schedule_priority capability is required to change the value ->
https://docs.splunk.com/Documentation/Splunk/8.0.6/RESTREF/RESTsearch#saved.2Fsearches

[igaskin]

@igaskin - I don't see the same issue when using the example.tf in the repo as an admin user. Does the user have proper permissions to make the schedule_priority change in your case?
The provider basically uses the rest endpoints provided by Splunk and the REST API documents suggests that the edit_search_schedule_priority capability is required to change the value ->
https://docs.splunk.com/Documentation/Splunk/8.0.6/RESTREF/RESTsearch#saved.2Fsearches

I can work to get an example of a saved_search that will produce this error, the code I am working with will reproduce this error every time, which forces a destroy to update the resource. Unless I'm mistake the 400 Bad Request error is not due to a permission error, but rather a malformed input. If this is a permission error, and is in fact working as intended, the maybe we can move the discussion in the direction of providing more descriptive error messages, such as a 403 permission denied for permission errors.

[anushjay]

Thanks @igaskin . Hmm, a destroy should only happen when a required parameter is being changed. Yes, a sample of the saved_search resource that's being created and the field that is being updated will be great to have. In the meantime, we will try to look into what is going on our side.
I did notice that the error was permission related when the user level permissions did not support the action.

[jfelle-tenable]

I also have this issue. I made a basic saved search:

resource "splunk_saved_searches" "test_search" {
    cron_schedule                           = "0 * * * *"
    description                             = "Bad things happened"
    is_scheduled                            = true
    is_visible                              = true
    name                                    = "TestSearch"
    search                                  = <<-EOT
                                            my fun search was here
                                            EOT

    acl {
        app              = "search"
        can_change_perms = true
        can_share_app    = true
        can_share_global = true
        can_share_user   = true
        can_write        = true
        owner            = "jfelle"
        read             = [ "*",
        ]
        removable        = true
        sharing          = "app"
        write            = [
            "*",
        ]
    }
}

The added some alert actions and saw the problem

resource "splunk_saved_searches" "test_search" {
    action_slack_param_attachment           = "alert_link"
    action_slack_param_channel              = "#dev_null"
    action_slack_param_message              = "Bad things happened"
    actions                                 = "slack"
    alert_comparator                        = "greater than"
    alert_expires                           = "24h"
    alert_severity                          = 4
    cron_schedule                           = "0 * * * *"
    description                             = "Bad things happened"
    is_scheduled                            = true
    is_visible                              = true
    name                                    = "TestSearch"
    search                                  = <<-EOT
                                            my fun search was here
                                            EOT

    acl {
        app              = "search"
        can_change_perms = true
        can_share_app    = true
        can_share_global = true
        can_share_user   = true
        can_write        = true
        owner            = "jfelle"
        read             = [ "*",
        ]
        removable        = true
        sharing          = "app"
        write            = [
            "*",
        ]
    }
}

I am doing this from an admin role.

[karsov]

Requiring admin privileges for an update while they are not needed on create and when working with the web interface is a bug for me.
Furthermore my team cannot use an admin user and has a user that has the minimal necessary permissions for managing saved searches (works fine in the UI and the Splunk API).
The only really working option - deleting and re-creating all saved searches is not an option when one has a hundred saved searches.

That's why I've implemented a workaround in which only the modified fields of a saved search are sent to the API by the Splunk Terraform provider. It's only a workaround and I'm not promoting it as a PR because a proper solution would require a serious rewrite of the saved searches code in the provider.

In order to use it one has to build it and configure terraform to use a local provider.

https://github.com/splunk/terraform-provider-splunk/compare/master...karsov:fix/issue-2-saved-search-update?expand=1

[ftieben]

We are facing the same error and the workaround we are using is to group alerts into a module and recreate (destroy & apply) if there are bigger changes.

[anshulgargamway]

I am also facing the same issue !!
Any Idea when the proper fix will be rolled out for same.

[ftieben]

ok, so here is whats happening:

in order to set or update the schedule_priority field you need to have the have the edit_search_schedule_priority capability in Splunk. If you dont have this, your requests to the api are not allowed to even have this field.

If you dont set it in terraform, it will be left empty therefor Splunk is setting it to "default" on its own. Terraform will read this default value and write it back into the state. If there is now any update on the search, the update requests will contain schedule_priority="default". This request will fail now because you dont have the edit_search_schedule_priority capability.

I think simply not saving this parameter into the state should fix it. I'll create a PR for that.

[dtivadar1]

Hello everyone.

When I create a report with terraform, a chart also appears at the top of the report. Does anyone know which option applies to this and how to turn it off?

[micahkemp-splunk]

ok, so here is whats happening:

in order to set or update the schedule_priority field you need to have the have the edit_search_schedule_priority capability in Splunk. If you dont have this, your requests to the api are not allowed to even have this field.

If you dont set it in terraform, it will be left empty therefor Splunk is setting it to "default" on its own. Terraform will read this default value and write it back into the state. If there is now any update on the search, the update requests will contain schedule_priority="default". This request will fail now because you dont have the edit_search_schedule_priority capability.

I think simply not saving this parameter into the state should fix it. I'll create a PR for that.

A resource should be entirely managed by Terraform or entirely unmanaged by Terraform.

If the managed resource needs schedule_priority, the account Terraform uses needs to have that capability. Creating it in Terraform and editing it locally isn't something that will give you long term success for many reasons. Terraform should detect drift and revert it. Local changes getting ignored violates that concept.

If the issue described here is truly due to permissions/capabilities, the solution is also with permissions/capabilities, and not changes to the provider to ignore certain settings if/when the Terraform account doesn't have sufficient permissions to perform the operation.

[vgopensource]

I think the issue seems to be with the incorrect saved search URL endpoint for a user acl.

if you see here we are passing the values

endpoint := client.BuildSplunkURL(nil, "servicesNS", owner, app, "saved", "searches", name)

terraform-provider-splunk/client/saved_searches.go

Line 41 in f60ef19

endpoint := client.BuildSplunkURL(nil, "servicesNS", owner, app, "saved", "searches", name)

REST API endpoint that worked for me was:

curl -vv -k -H "Authorization: Splunk <Token> " https://local:8089/servicesNS/nobody/testapp/saved/searches/testquery -d search="index test123"

Now owner as nobody seems allowing the non admin user to update the query if your app is shared and owned by you.

[rayanazzam]

Is there a version that fixes this issue?

[ohmakai]

My team is experiencing this issue as well. The first TF apply works fine without issue. Any consecutive TF apply fails with:

Error: 400 Bad Request: {"messages":[{"type":"ERROR","text":"Argument "auto_summarize.command" is not supported by this handler."}]}

None of the changes we saw in the TF files were anything more than minor value changes (e.g. modifying the cron schedule), so they shouldn't have caused any error.

We narrowed it down to something in changes made in v1.4.20. For the time being, we've asked people experiencing this issue to use v1.4.19, which works as expected.

[seth-carroll]

The difference between 1.4.19 and 1.4.20 is that it now bubbles up the 400 error to Terraform. Previously Terraform was acting like the update was applied when it actually wasn't. In the index=_internal sourcetype=splunkd_access logs, we could see the 400 error even in the older TF provider versions.

For us, the fix was to add the accelerate_search capability to the Splunk role used by our Terraform CI/CD pipeline. It seems that the Terraform provider makes an API call requiring that capability regardless if the saved search is accelerated.

I added a comment in #127 which seems to be a duplicate of this issue. For reference, the full list of capabilities in our role is listed in this comment on #127.

Hopefully this is helpful for others running into this issue.

[prudhvich0412]

Hi, I am also facing a similar issue. I have created 2 users one is terraform_plan(ReadOnly capabilities) and used when you raise a Merge request and another is terraform_apply(admin capabilities) leveraged upon merge to the branch, i was able to create the saved_search for the first time without any issue, both terraform plan and terraform apply are successful as these resources did not exist before. When i tried to make an update to the saved_search i created that's when terraform plan failed with 403. After 2 months of troubleshooting , i came to know that the issue in my case is due to the owner field in the acl. When i provide a owner(xxxx) and run the terraform plan with terraform_plan user, the provider goes into the particular user folder/namespace to read the saved_search(in this case which is owned by xxxx and terraform_plan doesn't have access to it with Read only capabilities.) so i had to change the owner from xxxx to Nobody to let the provider know that all the users have read only access to the saved searches and instead of going into a folder/namespace endpoint as mentioned here

terraform-provider-splunk/client/saved_searches.go

Line 41 in f60ef19

endpoint := client.BuildSplunkURL(nil, "servicesNS", owner, app, "saved", "searches", name)

it ignores the servicesNS and reads the saved search successfully.

That resolved the issue for me but i cannot use Nobody user because we have some custom configuration given to a role assigned to xxxx user like increasing srchDiskQuota, etc... which cannot be achieved by using Nobody user.

I just wanted to post this here so if anyone runs into similar issue it could be helpful.