Question 111004 in Json Question posted in
Our archive of expertly curated questions and answers provides insights and solutions to common problems related to this popular data interchange format. From parsing and manipulating JSON data to integrating it with various programming languages and web services, our archive has got you covered. Start exploring today and take your JSON skills to the next level

Powershell – Cannot parse JSON – Get NotSpecified error

Code used to try parsing file

$myJson = Get-Content cleantest2.json -Raw | ConvertFrom-Json

Get below error when trying to parse JSON file.

$myJson = Get-Content cleantest2.json -Raw | ConvertFrom-Json
+                                              ~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [ConvertFrom-Json], ArgumentException
    + FullyQualifiedErrorId : System.ArgumentException,Microsoft.PowerShell.Commands.ConvertFromJsonCommand

EDIT: As noted, the above error was truncated because it was too long. Here is the full error.

ConvertFrom-Json : Invalid object passed in, ':' or '}' expected. (2): {"id":0,"resourcePackage
s":[{"resourcePackage":{"name":"SharedResources","type":2,"i
tems":[{"type":202,"path":"BaseThemes/CY18SU07.json","name"
:"CY18SU07"}],"disabled":false}}],"sections":[{"id":0,"name"
:"ReportSection","displayName":"Page 1","filters":"[]","ord
inal":0,"visualContainers":[{"x":10,"y":0,"z":1,"width":280,
"height":280,"config":"{"name":"5a0b8ef6c97162087ff7",
"layouts":[{"id":0,"position":{"x":10,"y":0,"width
":280,"height":280,"z":1}}],"singleVisual":{"visual
Type":"tableEx","projections":{"Values":[{"queryRef
":"servers_and_applications.App Name"},{"queryRef":"se
rvers_and_applications.Cust_Nme"},{"queryRef":"servers_a
nd_applications.MU_ID"}]},"prototypeQuery":{"Version":
2,"From":[{"Name":"s","Entity":"servers_and_applica
tions"}],"Select":[{"Column":{"Expression":{"Source
Ref":{"Source":"s"}},"Property":"App
Name"},"Name":"servers_and_applications.App Name"},{"
Column":{"Expression":{"SourceRef":{"Source":"s"}},
"Property":"Cust_Nme"},"Name":"servers_and_applicati
ons.Cust_Nme"},{"Column":{"Expression":{"SourceRef":{
"Source":"s"}},"Property":"MU_ID"},"Name":"serve
rs_and_applications.MU_ID"}]},"drillFilterOtherVisuals":t
rue}}","filters":"[]","query":"{"Commands":[{"SemanticQu
eryDataShapeCommand":{"Query":{"Version":2,"From":[{
"Name":"s","Entity":"servers_and_applications"}],"S
elect":[{"Column":{"Expression":{"SourceRef":{"Sourc
e":"s"}},"Property":"App
Name"},"Name":"servers_and_applications.App Name"},{"
Column":{"Expression":{"SourceRef":{"Source":"s"}},
"Property":"Cust_Nme"},"Name":"servers_and_applicati
ons.Cust_Nme"},{"Column":{"Expression":{"SourceRef":{
"Source":"s"}},"Property":"MU_ID"},"Name":"serve
rs_and_applications.MU_ID"}]},"Binding":{"Primary":{"G
roupings":[{"Projections":[0,1,2],"Subtotal":1}]},"Da
taReduction":{"DataVolume":3,"Primary":{"Window":{"C
ount":500}}},"Version":1}}}]}","dataTransforms":"{"proj
ectionOrdering":{"Values":[0,1,2]},"queryMetadata":{"S
elect":[{"Restatement":"App
Name","Name":"servers_and_applications.App Name","Typ
e":2048},{"Restatement":"Cust_Nme","Name":"servers_a
nd_applications.Cust_Nme","Type":2048},{"Restatement":
"MU_ID","Name":"servers_and_applications.MU_ID","Type
":2048}]},"visualElements":[{"DataRoles":[{"Name":"
Values","Projection":0,"isActive":false},{"Name":"Va
lues","Projection":1,"isActive":false},{"Name":"Val
ues","Projection":2,"isActive":false}]}],"selects":[{
"displayName":"App
Name","queryName":"servers_and_applications.App Name",
"roles":{"Values":true},"type":{"category":null,"un
derlyingType":1},"expr":{"Column":{"Expression":{"S
ourceRef":{"Entity":"servers_and_applications"}},"Prop
erty":"App Name"}}},{"displayName":"Cust_Nme","quer
yName":"servers_and_applications.Cust_Nme","roles":{"V
alues":true},"type":{"category":null,"underlyingType
":1},"expr":{"Column":{"Expression":{"SourceRef":{"
Entity":"servers_and_applications"}},"Property":"Cust
_Nme"}}},{"displayName":"MU_ID","queryName":"servers
_and_applications.MU_ID","roles":{"Values":true},"typ
e":{"category":null,"underlyingType":1},"expr":{"Col
umn":{"Expression":{"SourceRef":{"Entity":"servers_
and_applications"}},"Property":"MU_ID"}}}]}"}],"config"
:"{}","displayOption":1,"width":1280,"height":720}],"config
":"{"version":"3.19","themeCollection":{"baseTheme":
{"name":"CY18SU07","version":"3.22","type":2}},"
activeSectionIndex":0,"defaultDrillFilterOtherVisuals":tr
ue,"settings":{"useStylableVisualContainerHeader":true,
"exportDataMode":1}}","layoutOptimization":0}
At line:1 char:46
+ $myJson = Get-Content cleantest2.json -Raw | ConvertFrom-Json
+                                              ~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [ConvertFrom-Json], ArgumentException
    + FullyQualifiedErrorId : System.ArgumentException,Microsoft.PowerShell.Commands.ConvertFromJsonCommand

Below is file that was trying to get parsed.

{"id":0,"resourcePackages":[{"resourcePackage":{"name":"SharedResources","type":2,"items":[{"type":202,"path":"BaseThemes/CY18SU07.json","name":"CY18SU07"}],"disabled":false}}],"sections":[{"id":0,"name":"ReportSection","displayName":"Page 1","filters":"[]","ordinal":0,"visualContainers":[{"x":10,"y":0,"z":1,"width":280,"height":280,"config":"{"name":"5a0b8ef6c97162087ff7","layouts":[{"id":0,"position":{"x":10,"y":0,"width":280,"height":280,"z":1}}],"singleVisual":{"visualType":"tableEx","projections":{"Values":[{"queryRef":"servers_and_applications.App Name"},{"queryRef":"servers_and_applications.Cust_Nme"},{"queryRef":"servers_and_applications.MU_ID"}]},"prototypeQuery":{"Version":2,"From":[{"Name":"s","Entity":"servers_and_applications"}],"Select":[{"Column":{"Expression":{"SourceRef":{"Source":"s"}},"Property":"App Name"},"Name":"servers_and_applications.App Name"},{"Column":{"Expression":{"SourceRef":{"Source":"s"}},"Property":"Cust_Nme"},"Name":"servers_and_applications.Cust_Nme"},{"Column":{"Expression":{"SourceRef":{"Source":"s"}},"Property":"MU_ID"},"Name":"servers_and_applications.MU_ID"}]},"drillFilterOtherVisuals":true}}","filters":"[]","query":"{"Commands":[{"SemanticQueryDataShapeCommand":{"Query":{"Version":2,"From":[{"Name":"s","Entity":"servers_and_applications"}],"Select":[{"Column":{"Expression":{"SourceRef":{"Source":"s"}},"Property":"App Name"},"Name":"servers_and_applications.App Name"},{"Column":{"Expression":{"SourceRef":{"Source":"s"}},"Property":"Cust_Nme"},"Name":"servers_and_applications.Cust_Nme"},{"Column":{"Expression":{"SourceRef":{"Source":"s"}},"Property":"MU_ID"},"Name":"servers_and_applications.MU_ID"}]},"Binding":{"Primary":{"Groupings":[{"Projections":[0,1,2],"Subtotal":1}]},"DataReduction":{"DataVolume":3,"Primary":{"Window":{"Count":500}}},"Version":1}}}]}","dataTransforms":"{"projectionOrdering":{"Values":[0,1,2]},"queryMetadata":{"Select":[{"Restatement":"App Name","Name":"servers_and_applications.App Name","Type":2048},{"Restatement":"Cust_Nme","Name":"servers_and_applications.Cust_Nme","Type":2048},{"Restatement":"MU_ID","Name":"servers_and_applications.MU_ID","Type":2048}]},"visualElements":[{"DataRoles":[{"Name":"Values","Projection":0,"isActive":false},{"Name":"Values","Projection":1,"isActive":false},{"Name":"Values","Projection":2,"isActive":false}]}],"selects":[{"displayName":"App Name","queryName":"servers_and_applications.App Name","roles":{"Values":true},"type":{"category":null,"underlyingType":1},"expr":{"Column":{"Expression":{"SourceRef":{"Entity":"servers_and_applications"}},"Property":"App Name"}}},{"displayName":"Cust_Nme","queryName":"servers_and_applications.Cust_Nme","roles":{"Values":true},"type":{"category":null,"underlyingType":1},"expr":{"Column":{"Expression":{"SourceRef":{"Entity":"servers_and_applications"}},"Property":"Cust_Nme"}}},{"displayName":"MU_ID","queryName":"servers_and_applications.MU_ID","roles":{"Values":true},"type":{"category":null,"underlyingType":1},"expr":{"Column":{"Expression":{"SourceRef":{"Entity":"servers_and_applications"}},"Property":"MU_ID"}}}]}"}],"config":"{}","displayOption":1,"width":1280,"height":720}],"config":"{"version":"3.19","themeCollection":{"baseTheme":{"name":"CY18SU07","version":"3.22","type":2}},"activeSectionIndex":0,"defaultDrillFilterOtherVisuals":true,"settings":{"useStylableVisualContainerHeader":true,"exportDataMode":1}}","layoutOptimization":0}

Reformatted:

{
    "id":  0,
    "resourcePackages":  [
                             {
                                 "resourcePackage":  {
                                                         "name":  "SharedResources",
                                                         "type":  2,
                                                         "items":  [
                                                                       {
                                                                           "type":  202,
                                                                           "path":  "BaseThemes/CY18SU07.json",
                                                                           "name":  "CY18SU07"
                                                                       }
                                                                   ],
                                                         "disabled":  false
                                                     }
                             }
                         ],
    "sections":  [
                     {
                         "id":  0,
                         "name":  "ReportSection",
                         "displayName":  "Page 1",
                         "filters":  "[]",
                         "ordinal":  0,
                         "visualContainers":  [
                                                  {
                                                      "x":  10,
                                                      "y":  0,
                                                      "z":  1,
                                                      "width":  280,
                                                      "height":  280,
                                                      "config":  "{"name":"5a0b8ef6c97162087ff7","layouts":[{"id":0,"position":{"x":10,"y":0,"width":280,"height":280,"z":1}}],"singleVisual":{"visualType":"tableEx","projections":{"Values":[{"queryRef":"servers_and_applications.App Name"},{"queryRef":"servers_and_applications.Cust_Nme"},{"queryRef":"servers_and_applications.MU_ID"}]},"prototypeQuery":{"Version":2,"From":[{"Name":"s","Entity":"servers_and_applications"}],"Select":[{"Column":{"Expression":{"SourceRef":{"Source":"s"}},"Property":"App Name"},"Name":"servers_and_applications.App Name"},{"Column":{"Expression":{"SourceRef":{"Source":"s"}},"Property":"Cust_Nme"},"Name":"servers_and_applications.Cust_Nme"},{"Column":{"Expression":{"SourceRef":{"Source":"s"}},"Property":"MU_ID"},"Name":"servers_and_applications.MU_ID"}]},"drillFilterOtherVisuals":true}}",
                                                      "filters":  "[]",
                                                      "query":  "{"Commands":[{"SemanticQueryDataShapeCommand":{"Query":{"Version":2,"From":[{"Name":"s","Entity":"servers_and_applications"}],"Select":[{"Column":{"Expression":{"SourceRef":{"Source":"s"}},"Property":"App Name"},"Name":"servers_and_applications.App Name"},{"Column":{"Expression":{"SourceRef":{"Source":"s"}},"Property":"Cust_Nme"},"Name":"servers_and_applications.Cust_Nme"},{"Column":{"Expression":{"SourceRef":{"Source":"s"}},"Property":"MU_ID"},"Name":"servers_and_applications.MU_ID"}]},"Binding":{"Primary":{"Groupings":[{"Projections":[0,1,2],"Subtotal":1}]},"DataReduction":{"DataVolume":3,"Primary":{"Window":{"Count":500}}},"Version":1}}}]}",
                                                      "dataTransforms":  "{"projectionOrdering":{"Values":[0,1,2]},"queryMetadata":{"Select":[{"Restatement":"App Name","Name":"servers_and_applications.App Name","Type":2048},{"Restatement":"Cust_Nme","Name":"servers_and_applications.Cust_Nme","Type":2048},{"Restatement":"MU_ID","Name":"servers_and_applications.MU_ID","Type":2048}]},"visualElements":[{"DataRoles":[{"Name":"Values","Projection":0,"isActive":false},{"Name":"Values","Projection":1,"isActive":false},{"Name":"Values","Projection":2,"isActive":false}]}],"selects":[{"displayName":"App Name","queryName":"servers_and_applications.App Name","roles":{"Values":true},"type":{"category":null,"underlyingType":1},"expr":{"Column":{"Expression":{"SourceRef":{"Entity":"servers_and_applications"}},"Property":"App Name"}}},{"displayName":"Cust_Nme","queryName":"servers_and_applications.Cust_Nme","roles":{"Values":true},"type":{"category":null,"underlyingType":1},"expr":{"Column":{"Expression":{"SourceRef":{"Entity":"servers_and_applications"}},"Property":"Cust_Nme"}}},{"displayName":"MU_ID","queryName":"servers_and_applications.MU_ID","roles":{"Values":true},"type":{"category":null,"underlyingType":1},"expr":{"Column":{"Expression":{"SourceRef":{"Entity":"servers_and_applications"}},"Property":"MU_ID"}}}]}"
                                                  }
                                              ],
                         "config":  "{}",
                         "displayOption":  1,
                         "width":  1280,
                         "height":  720
                     }
                 ],
    "config":  "{"version":"3.19","themeCollection":{"baseTheme":{"name":"CY18SU07","version":"3.22","type":2}},"activeSectionIndex":0,"defaultDrillFilterOtherVisuals":true,"settings":{"useStylableVisualContainerHeader":true,"exportDataMode":1}}",
    "layoutOptimization":  0
}

Items attempted:

  • Used online formator on text and then re-ran command on resulting string. – Worked
  • Tried to import raw string and export as JSON using ConverTo-Json method but didn’t work correctly.
  • Tried parsing using Python but had similar issue.
Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    I was able to fix this by changing the encoding to unicode. See below sample execution.

    $myJson = Get-Content cleantest2.json -Encoding unicode | ConvertFrom-Json

  2. 0

    Your own answer implies that your input file, cleantest2.json, was malformed:

    • The fact that -Encoding Unicode solved your problem implies that the file is a UTF-16LE-encoded file that lacks a BOM, which means that it provides no indication to readers of the file what its encoding is, and causes Get-Content to (justifiably) misinterpret the file.

    • Consider correcting the problem at the source: make the creator of file cleantest2.json create it with the appropriate BOM, the way that Set-Content -Encoding Unicode does, for instance.

    As for troubleshooting your symptom:

    If a string’s display output is seemingly correct, but processing of the string doesn’t work as expected, the cause is usually the presence of hidden control characters.

    Specifically, misreading a BOM-less UTF16-LE ("Unicode") file as either ANSI-encoded (which Windows PowerShell does by default) or UTF-8 (as PowerShell (Core) 7+ does by default) results in all ASCII-range characters being followed by a NUL character (code point 0x0)each, and these NUL characters are invisible when you print the string.

    For testing, you can create a malformed UTF-16LE file as follows:

    # Write 'foo' with UTF-16LE encoding to file 'cleantest2.json' *without a BOM*:
[IO.File]::WriteAllText("$PWD/cleantest2.json", "f`0o`0`o`0")

    A simple test for the presence of at least one NUL character (expressed as "`0" in PowerShell) is:

    (Get-Content cleantest2.json -Raw) -match "`0"  # "`0" creates a NUL

    Piping to Format-Hex can also help in this case:[1]

    # Look for `00` values.
Get-Content cleantest2.json -Raw | Format-Hex

    The Debug-String helper function, available from this MIT-licensed Gist offers a direct visualization:

    # Download and define the Debug-String function.
# NOTE: 
#  I can personally assure you that doing this is safe, but you
#  you should always check the source code first.
irm https://gist.github.com/mklement0/7f2f1e13ac9c2afaf0a0906d08b392d1/raw/Debug-String.ps1 | iex

Get-Content cleantest2.json -Raw | Debug-String

    Output:

    screenshot

    A note re spotting the problem with the help of text editors:

    • This may not help, because many editors correctly detect UTF-16LE files even without a BOM (e.g., Notepad, Visual Studio Code), and in their status bar report just "UTF-16 LE" whether or not a BOM is present.

    • However, if an editor offers a hex view (of the bytes that make up the file), you’ll be able to tell: If an UTF-16LE file does have a BOM, the first two bytes are FF FE (for UTF-8 files that have a BOM, the first three bytes are EF BB BF).

      • Visual Studio Code has a Hex Editor extension

      • js2010 reports that Emacs has a hexl-mode.

    [1] Use of Format-Hex to discover (non-ASCII-range) Unicode characters isn’t an option in Windows PowerShell, because it represents any non-ASCII chars. as 3f, i.e. literal ?. In PowerShell (Core) it shows the ASCII rendering of the individual bytes that make up the UTF-8 encoding of non-ASCII characters, which can be confusing.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search