FileHooks: Dokumentation optimieren

Ich verstehe die Dokumentation, die mit https://extgit.iaik.tugraz.at/codeability/codeability/sharingplattform/file-hooks/-/commit/1c6158d3184c0c445c787db059a519266c02f489 eingeführt wurde nicht :-(

Eigentlich möchte ich, falls der Index aus irgendeinem Grund korrupt ist (z.B. weil wegen Racing-Conditions MetaDaten zweimal indexiert wurden).

1. Was muss ich dann genau aufrufen?
2. Wie kann ich das als Administrator ggf. aus der Suchplattform triggern?

changed milestone to %2021 March Sprint Sharing Plattform

Noch eine Ergänzung: Ich habe versucht, die Volltextindexierung nochmals anzustossen mit

• python3 cli.py create-index -idx-fulltext idx_fulltext_1 -idx-metadata idx_metadata_1
• python3 cli.py switch-alias -old-idx-fulltext idx_fulltext_0 -new-idx-fulltext idx_fulltext_1 -old-idx-metadata idx_metadata_0 -new-idx-metadata idx_metadata_1

Ergebnis: Es sind immer noch Duplikate im Index.

Currently, there is no GUI. It is just a CLI. The commands you posted seem to be correct for the purpose of creating a new index and using it.

I am not sure what you mean by duplicates. Can you provide an example?

added 1 design

Bei der Volltextsuche nach "100" wird das Projekt "Meta Test Data 100" zweimal gefunden.

Vermutlich wurde das Projekt irgendwie zweimal indiziert.

Ich möchte den Index wieder von Grund auf aufbauen. D.h. alle (relevanten) Projekte im Repository neu indizieren.

Oh sorry, das Problem hat sich geklärt. Es sind Metadaten aus zwei Projekten 167 und 169.

added 1 design

By design it is currently not possible to get a duplicated entry in the metadata index for a repository, since the document id (_id) is based on the project id (from GitLab, project.project_id). Consequently, only one metadata file is allowed for a project (This might change in future, when collections are supported?).

As you already mentioned, there are 2 documents with title Meta Test Data 100.

curl -X GET "localhost:9200/metadata/_search?pretty" -H 'Content-Type: application/json' -d '
{
  "query": {
    "term": {
      "metadata.title": "100" 
    }
  }
}
'

Currently yields the following result:

{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 2,
    "max_score" : 1.900004,
    "hits" : [
      {
        "_index" : "idx_metadata_1",
        "_type" : "_doc",
        "_id" : "167",
        "_score" : 1.900004,
        "_source" : {
          "metadata" : {
            "metadataVersion" : "0.2",
            "type" : "collection",
            "collectionContent" : [
              "src/if-001/if-001.yml",
              "src/if-002/if-002.yml",
              "src/if-003/if-003.yml"
            ],
            "identifier" : "javaCourseTUWienTest",
            "structure" : "hierarchical",
            "version" : "1.0",
            "status" : "final",
            "title" : "Meta Test Data 100",
            "description" : "Mein test für die time required. (set back)!",
            "programmingLanguage" : [
              "JAVA"
            ],
            "language" : [
              "de"
            ],
            "educationLevel" : "Anfänger, (to be detailed)",
            "audience" : "Anfaenger",
            "keyword" : [
              "Java",
              "IOTest",
              "latex",
              "testing100"
            ],
            "license" : "MIT",
            "creator" : [
              {
                "name" : "Stefan Podlipnig",
                "affiliation" : "TU Wien",
                "email" : "stefan.podlipnig@tuwien.ac.at"
              }
            ],
            "contributor" : [
              {
                "name" : "Daniel Bastta",
                "affiliation" : "TU Wien",
                "email" : "daniel.bastta@tuwien.ac.at"
              },
              {
                "name" : "Andreas Merckel",
                "affiliation" : "TU Wien",
                "email" : "andreas.merkel@tuwien.ac.at"
              },
              {
                "name" : "Kerstin Limbeck",
                "affiliation" : "TU Wien",
                "email" : "kerstin.limbeck@tuwien.ac.at"
              }
            ],
            "publisher" : [
              {
                "name" : "Andreas Merckel",
                "affiliation" : "TU Wien",
                "email" : "andreas.merkel@tuwien.ac.at"
              }
            ],
            "format" : [
              "latex"
            ],
            "deprecated" : false,
            "difficulty" : "simple",
            "valid" : {
              "start" : "2020-01-01",
              "end" : "2030-12-31"
            },
            "requires" : [
              "Java14"
            ],
            "timeRequired" : "2:00",
            "image" : "./courseIcon.png"
          },
          "project" : {
            "project_id" : 167
          }
        }
      },
      {
        "_index" : "idx_metadata_1",
        "_type" : "_doc",
        "_id" : "169",
        "_score" : 1.5809193,
        "_source" : {
          "metadata" : {
            "metadataVersion" : "0.2",
            "type" : "collection",
            "collectionContent" : [
              "src/if-001/if-001.yml",
              "src/if-002/if-002.yml",
              "src/if-003/if-003.yml"
            ],
            "identifier" : "javaCourseTUWienTest",
            "structure" : "hierarchical",
            "version" : "1.0",
            "status" : "final",
            "title" : "Meta Test Data 100 (Projekt test_lk)",
            "description" : "Mein test für die time required. Hello World2!",
            "programmingLanguage" : [
              "JAVA"
            ],
            "language" : [
              "de"
            ],
            "educationLevel" : "Anfänger, (to be detailed)",
            "audience" : "Anfaenger",
            "keyword" : [
              "Java",
              "IOTest",
              "latex",
              "testing100"
            ],
            "license" : "MIT",
            "creator" : [
              {
                "name" : "Stefan Podlipnig",
                "affiliation" : "TU Wien",
                "email" : "stefan.podlipnig@tuwien.ac.at"
              }
            ],
            "contributor" : [
              {
                "name" : "Daniel Bastta",
                "affiliation" : "TU Wien",
                "email" : "daniel.bastta@tuwien.ac.at"
              },
              {
                "name" : "Andreas Merckel",
                "affiliation" : "TU Wien",
                "email" : "andreas.merkel@tuwien.ac.at"
              },
              {
                "name" : "Kerstin Limbeck",
                "affiliation" : "TU Wien",
                "email" : "kerstin.limbeck@tuwien.ac.at"
              }
            ],
            "publisher" : [
              {
                "name" : "Andreas Merckel",
                "affiliation" : "TU Wien",
                "email" : "andreas.merkel@tuwien.ac.at"
              }
            ],
            "format" : [
              "latex"
            ],
            "deprecated" : false,
            "difficulty" : "simple",
            "valid" : {
              "start" : "2020-01-01",
              "end" : "2030-12-31"
            },
            "requires" : [
              "Java14"
            ],
            "timeRequired" : "2:00",
            "image" : "./courseIcon.png"
          },
          "project" : {
            "project_id" : 169
          }
        }
      }
    ]
  }
}

An update for the cli documentation has been pushed to the development branch.

assigned to @Michael.Breu and unassigned @lukas.kaltenbrunner

OK, https://extgit.iaik.tugraz.at/codeability/codeability/sharingplattform/file-hooks/-/commit/1c6158d3184c0c445c787db059a519266c02f489 passt .

closed