upgrade-cas-es-issues.md

0. Warnings during install phase (Nocaching TTL defined, using default of 30sec)
1. Installation failed to upload initial content. Had to do this separately.
2. x-info doesn't work (fixed by https://github.com/CM-Well/CM-Well/commit/2dcda7153439f7ff86f2d4da801381b67d5a636c)
3. stream failing to fetch all results.
4. Upgrade capability should be disabled for complete version.
5. Stats api doesn't work - Fixed, but changes default behavior of :: vs : need documentation
6. vm.max_map_count was setup for single elastic on a machine to pass install check. Does it need to be increased for 4 machines? Eli will ask Dudi.
7. x-info will show up to 10000 versions of an infoton. https://github.com/CM-Well/CM-Well/blob/master/docs/Admin.TestAndFixCorruptedData.md should be updated that x-info fetches all versions, unless there's more than 10K.
8. x-fix doesn't work, return with status code 500
9. jsonl formatted infotons return with fields in different orders - we must fix our tests to be less specific (don't expect id property to come before lastModified for example).
10. _all API doesn't work. Always returns 0 results.
11. prepare machines doesn't delete sshpass postinstall
12. Ingesting any update to infoton creates a duplicate - BLOCKER - issue with crawler. fixed by Eli.
13. Disable printing akka-config details in the logs.
14. DC Sync doesn't work. fails to fetch any data. Fixed.
15. Installation has issue with proxy settings (fails to complete if proxy is set up - disabling proxy helps to workaround)
16. DC Sync doesn't create parent with indexTime=0 (permid root).
17. DC Sync fails to start if qp contains system.lastModified>1514764800000, internally an exception is thrown: failed to parse date field [1514764800000] with format [yyyy-MM-dd'T'HH:mm:ss.SSS'Z']
18. System.lastModified field can be referred only using correct dateTime format. like this: /?op=search&recursive&qp=system.lastModified%3E2018-01-01T00:00:00.000Zotherwise {"success":false, "error":"all shards failed"} is received. This is a functionality regression, which will affect multiple customers, if isn't fixed.
19. After install, any search request ends with 424, and a workaround is to ingest any infoton.
20. Ingest large infoton made bg to be stuck on exception from kafka org.apache.kafka.common.errors.RecordTooLargeException: The message is 1539912 bytes when serialized which is larger than the maximum . The partition gets filled fast afterwards and the environment is stuck.
21. Get nex chunk (create iterator) api doesn't work. In test data it pulls only 4 of expected 5 infotons.
22. Navigating to http://cm-well-host/health/es_status ends with error "invalid_index_name_exception"
23. Difference in displayed data when navigating http://cm-well-host/health/es_thread_pool
24. Error messages when navigating http://cm-well-host/health/es_status
25. Duplicate index time for /meta/xxx infotons which breaks ingests. This line in logs: java.lang.Exception: failed to getMetaFieldInfoton(HashedFieldKey(hasAction,wbGrMw)) from IRW
26. Stream of tsvs is really slow (120K/s) even when client is local.
27. Upgrade of minor version of Elastic have failed (from ES 6.6.0 to 6.6.2) on atom.
28. New Elastic requires higher version of Linux kernel than what we have on most on-prem environments.

• lastModified as epoch
• x-fix
• big number