Can t create file mapping что это

Cannot create file mapping for a file I just opened, get «path not found»

The file is created allright, but the value of hFileMapping is NULL ! Which is weird because I’m using hFile (which is a valid handle!) and when I check GetLastError() status after the call to CreateFileMApping() I’m getting ERROR_PATH_NOT_FOUND , which is weird, because another function just used it to create the file. Any ideas?

44.5k 11 11 gold badges 96 96 silver badges 135 135 bronze badges
asked Feb 1, 2012 at 13:16
5,484 4 4 gold badges 43 43 silver badges 68 68 bronze badges
Are you sure that the error is from that functions?
Feb 1, 2012 at 13:19

Yeap, when I’m checking it before that call I’m getting what’s set there by CreateFile() — ERROR_ALREADY_EXISTS (which is ok, it overwrites it and sets the error code to that)

File type associations

For language-specific features (such as syntax highlighting and code analysis ) in files representing different languages and technologies, IntelliJ IDEA maintains a list of file types , each of which links a language service with one or more filename patterns.

The default list of file types covers all relevant filename patterns, but you can add new file types for your custom language files and change the associated filename patterns for existing file types.

If you are working on a language that is not supported in IntelliJ IDEA by default, there might be plugins supporting that language depending on the IntelliJ IDEA edition.

When you open a file in the editor, IntelliJ IDEA chooses the file type and the corresponding language service according to the filename pattern. If the filename doesn’t match any of the patterns registered for file types, you can associate the filename pattern with a specific file type.

Apart from that, you can make IntelliJ IDEA the default application for opening specific file types from the file manager on your operating system.

Add a custom file type

If you work on a language that is not supported by default and there are no plugins for it, you can configure a simple language service for files associated with this language — you will enjoy syntax highlighting for keywords, comments, and braces and have some basic editor helpers such as adding line/block comments with Control+/ / Control+Shift+/ and extending/shrinking selection according to the structure with Control+W / Control+Shift+W .

1. Press Control+Alt+S to open the IDE settings and then select Editor | File Types .
2. In the Recognized File Types section, click , specify the name of the new type, and provide a description.
3. In the Syntax Highlighting section, configure case sensitivity, brace matching settings, and specify ways of defining comments:
   • Line comment : specify characters that indicate the beginning of a single-line comment.
   • Only at line start : characters that indicate the beginning of a line comment are recognized as a comment if they are located at the beginning of a line.
   • Block comment start , Block comment end : specify characters that indicate the beginning and the end of a block comment.
   • Hex prefix : specify characters that indicate that the subsequent value is a hexadecimal number (for example, 0x ).
   • Number postfixes : specify characters that indicate which numeric system or unit is used. A postfix is a trailing string of characters (for example, e-3, kg ).
   • Support paired braces , Support paired brackets , Support paired parens , Support string escapes : select these checkboxes to highlight paired braces, brackets, parentheses, and string escapes.
4. In the Keywords section, you can specify up to four lists of keywords. Keywords of each list will be highlighted differently in the editor and will be auto-completed.
5. The Ignore case checkbox indicates whether keywords in files of the custom format are case-sensitive.
6. You can customize colors for syntax highlighting of language-specific keywords, comments, and other identifiers on the Editor | Color Scheme | User-Defined File Types settings page.

In IntelliJ IDEA, recognized files are not always supplied with extensive support. For example, .php files are recognized in the IntelliJ IDEA Community Edition and marked with the corresponding icon, although this edition does not provide PHP development support.

Configure associations between filename patterns and file types

Associate a filename pattern with specific file type

1. If IntelliJ IDEA cannot identify the type of the file that you are trying to open or create, it displays the Register New File Type Association dialog where you can choose the way you want to process this file. If the dialog doesn’t appear automatically, right-click the file in the Project tool window and select Associate with File Type from the context menu or choose File | File Properties | Associate with File Type from the main menu.
2. In the Register New File Type Association dialog, select the necessary options:
3. From the File pattern list, select whether you want to specify a type for the current file ( file.extension ) or for all files with this extension ( *.extension ).
4. Select one of the following options:
   • Open matching files as text and auto-detect file type by content : open the file without an extension as a text file and identify its type by the content, for example, by the shebang line.
   • Open matching files in IntelliJ IDEA : associate the file with one of the existing file types. You can change this association later in the settings.
   • Open matching files in associated application : open the file in the default system application configured in your operating system. For example, .pdf files are opened in the default PDF viewer. If necessary, you can check and configure all filename patterns associated with system applications.
5. Click OK to apply the settings.

Change filename patterns associated with file type

1. Press Control+Alt+S to open the IDE settings and then select Editor | File Types .
2. From the Recognized File Types list, select the file type that you want to associate with other filename patterns.
3. Use the File name patterns section to make the necessary changes. You can add a new pattern (), remove an existing one (), or modify an existing pattern ().

If your project contains files in proprietary formats, such as .pdf and .docx , IntelliJ IDEA will open these files using the default application configured in your operating system. When a specific proprietary file format is not recognized, or you just want to open certain files with the system application, you can add the necessary associations.

Configure filename patterns associated with system applications

1. Press Control+Alt+S to open the IDE settings and then select Editor | File Types .
2. In the Recognized File Types list, select Files opened in associated applications .
3. In the File name patterns section on the right, click and specify a filename pattern that should be associated with an external application.

If a file is correctly associated with a specific file type by its filename pattern, but you want to process this file differently, you can override the file type association for this file only — other files matching that pattern will not be affected.

Override file type for specific file

1. In the Project tool window ( Alt+1 ), select one or more files that should have another file type association, right-click the selection and choose Override File Type .
2. From the list that opens, select a new file type. Use speed search to find the required file type faster.
3. To restore the original file type association according to the filename pattern, right-click the file or files again and select Revert File Type Override from the context menu.

Make IntelliJ IDEA the default app for specific file types

You can make IntelliJ IDEA the default application for opening specific file types from the default file manager on your operating system.

1. Press Control+Alt+S to open the IDE settings and then select Editor | File Types .
2. Click Associate File Types with IntelliJ IDEA and select the file extensions you want to open with the IDE.
3. Click OK and close the dialog. If you’re using macOS, restart your computer to apply the changes.

Ignore files and folders

IntelliJ IDEA also maintains a list of files and folders that are completely excluded from any kind of processing. Out of the box, this list includes temporary files, service files related to version control systems, and so on:

*.pyc;*.pyo;*.rbc;*.yarb;*~;.DS_Store;.git;.hg;.svn;CVS;__pycache__;_svn;vssver.scc;vssver2.scc;

Modify the list of ignored files and folders

1. Press Control+Alt+S to open the IDE settings and then select Editor | File Types .
2. Switch to the Ignored Files and Folders tab. You can add a new extension (), remove an existing one (), or modify an existing extension ().
3. Click Apply to save the changes and close the dialog.

Configure shebang commands for file types

IntelliJ IDEA can recognize file types by the path specified on the shebang line. A shebang is a combination of characters in a script file followed by a path to the interpreter program that should execute this script. It starts with #! and it’s always located on the first line of a script file.

1. Press Control+Alt+S to open the IDE settings and then select Editor | File Types .
2. From the Recognized File Types list, select the file type for which you want to configure a command.
3. In the HashBang patterns area, click ( Add HashBang Pattern ).
4. In the dialog that opens, specify the pattern that the IDE will use to recognize a file type, then click OK .
5. Click Apply to save the changes and close the dialog.

ConfigMaps

A ConfigMap is an API object used to store non-confidential data in key-value pairs. Pods can consume ConfigMaps as environment variables, command-line arguments, or as configuration files in a volume.

A ConfigMap allows you to decouple environment-specific configuration from your container images, so that your applications are easily portable.

Caution: ConfigMap does not provide secrecy or encryption. If the data you want to store are confidential, use a Secret rather than a ConfigMap, or use additional (third party) tools to keep your data private.

Motivation

Use a ConfigMap for setting configuration data separately from application code.

For example, imagine that you are developing an application that you can run on your own computer (for development) and in the cloud (to handle real traffic). You write the code to look in an environment variable named DATABASE_HOST . Locally, you set that variable to localhost . In the cloud, you set it to refer to a Kubernetes Service that exposes the database component to your cluster. This lets you fetch a container image running in the cloud and debug the exact same code locally if needed.

Note: A ConfigMap is not designed to hold large chunks of data. The data stored in a ConfigMap cannot exceed 1 MiB. If you need to store settings that are larger than this limit, you may want to consider mounting a volume or use a separate database or file service.

ConfigMap object

A ConfigMap is an API object that lets you store configuration for other objects to use. Unlike most Kubernetes objects that have a spec , a ConfigMap has data and binaryData fields. These fields accept key-value pairs as their values. Both the data field and the binaryData are optional. The data field is designed to contain UTF-8 strings while the binaryData field is designed to contain binary data as base64-encoded strings.

The name of a ConfigMap must be a valid DNS subdomain name.

Each key under the data or the binaryData field must consist of alphanumeric characters, — , _ or . . The keys stored in data must not overlap with the keys in the binaryData field.

Starting from v1.19, you can add an immutable field to a ConfigMap definition to create an immutable ConfigMap.

ConfigMaps and Pods

You can write a Pod spec that refers to a ConfigMap and configures the container(s) in that Pod based on the data in the ConfigMap. The Pod and the ConfigMap must be in the same namespace.

Note: The spec of a static Pod cannot refer to a ConfigMap or any other API objects.

Here’s an example ConfigMap that has some keys with single values, and other keys where the value looks like a fragment of a configuration format.

apiVersion: v1  kind: ConfigMap  metadata:  name: game-demo  data:  # property-like keys; each key maps to a simple value  player_initial_lives: "3"  ui_properties_file_name: "user-interface.properties"   # file-like keys  game.properties: |  enemy.types=aliens,monsters  player.maximum-lives=5  user-interface.properties: |  color.good=purple  color.bad=yellow  allow.textmode=true 

There are four different ways that you can use a ConfigMap to configure a container inside a Pod:

1. Inside a container command and args
2. Environment variables for a container
3. Add a file in read-only volume, for the application to read
4. Write code to run inside the Pod that uses the Kubernetes API to read a ConfigMap

These different methods lend themselves to different ways of modeling the data being consumed. For the first three methods, the kubelet uses the data from the ConfigMap when it launches container(s) for a Pod.

The fourth method means you have to write code to read the ConfigMap and its data. However, because you’re using the Kubernetes API directly, your application can subscribe to get updates whenever the ConfigMap changes, and react when that happens. By accessing the Kubernetes API directly, this technique also lets you access a ConfigMap in a different namespace.

Here’s an example Pod that uses values from game-demo to configure a Pod:

apiVersion: v1  kind: Pod  metadata:  name: configmap-demo-pod  spec:  containers:  - name: demo  image: alpine  command: ["sleep", "3600"]  env:  # Define the environment variable  - name: PLAYER_INITIAL_LIVES # Notice that the case is different here  # from the key name in the ConfigMap.  valueFrom:  configMapKeyRef:  name: game-demo # The ConfigMap this value comes from.  key: player_initial_lives # The key to fetch.  - name: UI_PROPERTIES_FILE_NAME  valueFrom:  configMapKeyRef:  name: game-demo  key: ui_properties_file_name  volumeMounts:  - name: config  mountPath: "/config"  readOnly: true  volumes:  # You set volumes at the Pod level, then mount them into containers inside that Pod  - name: config  configMap:  # Provide the name of the ConfigMap you want to mount.  name: game-demo  # An array of keys from the ConfigMap to create as files  items:  - key: "game.properties"  path: "game.properties"  - key: "user-interface.properties"  path: "user-interface.properties"  

A ConfigMap doesn’t differentiate between single line property values and multi-line file-like values. What matters is how Pods and other objects consume those values.

For this example, defining a volume and mounting it inside the demo container as /config creates two files, /config/game.properties and /config/user-interface.properties , even though there are four keys in the ConfigMap. This is because the Pod definition specifies an items array in the volumes section. If you omit the items array entirely, every key in the ConfigMap becomes a file with the same name as the key, and you get 4 files.

Using ConfigMaps

ConfigMaps can be mounted as data volumes. ConfigMaps can also be used by other parts of the system, without being directly exposed to the Pod. For example, ConfigMaps can hold data that other parts of the system should use for configuration.

The most common way to use ConfigMaps is to configure settings for containers running in a Pod in the same namespace. You can also use a ConfigMap separately.

For example, you might encounter addons or operators that adjust their behavior based on a ConfigMap.

Using ConfigMaps as files from a Pod

To consume a ConfigMap in a volume in a Pod:

1. Create a ConfigMap or use an existing one. Multiple Pods can reference the same ConfigMap.
2. Modify your Pod definition to add a volume under .spec.volumes[] . Name the volume anything, and have a .spec.volumes[].configMap.name field set to reference your ConfigMap object.
3. Add a .spec.containers[].volumeMounts[] to each container that needs the ConfigMap. Specify .spec.containers[].volumeMounts[].readOnly = true and .spec.containers[].volumeMounts[].mountPath to an unused directory name where you would like the ConfigMap to appear.
4. Modify your image or command line so that the program looks for files in that directory. Each key in the ConfigMap data map becomes the filename under mountPath .

This is an example of a Pod that mounts a ConfigMap in a volume:

apiVersion: v1  kind: Pod  metadata:  name: mypod  spec:  containers:  - name: mypod  image: redis  volumeMounts:  - name: foo  mountPath: "/etc/foo"  readOnly: true  volumes:  - name: foo  configMap:  name: myconfigmap 

Each ConfigMap you want to use needs to be referred to in .spec.volumes .

If there are multiple containers in the Pod, then each container needs its own volumeMounts block, but only one .spec.volumes is needed per ConfigMap.

Mounted ConfigMaps are updated automatically

When a ConfigMap currently consumed in a volume is updated, projected keys are eventually updated as well. The kubelet checks whether the mounted ConfigMap is fresh on every periodic sync. However, the kubelet uses its local cache for getting the current value of the ConfigMap. The type of the cache is configurable using the configMapAndSecretChangeDetectionStrategy field in the KubeletConfiguration struct. A ConfigMap can be either propagated by watch (default), ttl-based, or by redirecting all requests directly to the API server. As a result, the total delay from the moment when the ConfigMap is updated to the moment when new keys are projected to the Pod can be as long as the kubelet sync period + cache propagation delay, where the cache propagation delay depends on the chosen cache type (it equals to watch propagation delay, ttl of cache, or zero correspondingly).

ConfigMaps consumed as environment variables are not updated automatically and require a pod restart.

Note: A container using a ConfigMap as a subPath volume mount will not receive ConfigMap updates.

Immutable ConfigMaps

FEATURE STATE: Kubernetes v1.21 [stable]

The Kubernetes feature Immutable Secrets and ConfigMaps provides an option to set individual Secrets and ConfigMaps as immutable. For clusters that extensively use ConfigMaps (at least tens of thousands of unique ConfigMap to Pod mounts), preventing changes to their data has the following advantages:

• protects you from accidental (or unwanted) updates that could cause applications outages
• improves performance of your cluster by significantly reducing load on kube-apiserver, by closing watches for ConfigMaps marked as immutable.

You can create an immutable ConfigMap by setting the immutable field to true . For example:

apiVersion: v1  kind: ConfigMap  metadata:  .  data:  .  immutable: true 

Once a ConfigMap is marked as immutable, it is not possible to revert this change nor to mutate the contents of the data or the binaryData field. You can only delete and recreate the ConfigMap. Because existing Pods maintain a mount point to the deleted ConfigMap, it is recommended to recreate these pods.

What’s next

• Read about Secrets.
• Read Configure a Pod to Use a ConfigMap.
• Read about changing a ConfigMap (or any other Kubernetes object)
• Read The Twelve-Factor App to understand the motivation for separating code from configuration.

Feedback

Was this page helpful?

Thanks for the feedback. If you have a specific, answerable question about how to use Kubernetes, ask it on Stack Overflow. Open an issue in the GitHub Repository if you want to report a problem or suggest an improvement.

Saved searches

Use saved searches to filter your results more quickly

Cancel Create saved search

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session.

JuliaData / CSV.jl Public

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error: could not create file mapping: The operation comleted successfully #424

zjpi opened this issue May 6, 2019 · 13 comments

Error: could not create file mapping: The operation comleted successfully #424

zjpi opened this issue May 6, 2019 · 13 comments

Comments

zjpi commented May 6, 2019

I tried to read this example file

download("https://github.com/xiaodaigh/testing/raw/master/Performance_2003Q3.zip", "ok.zip") run(`unzip -o ok.zip`) using CSV path = "Performance_2003Q3.txt" @time a = CSV.read(path, delim = '|', header = 0);

I get the following error

ERROR: could not create file mapping: The operation completed successfully. Stacktrace: [1] error(::String) at .\error.jl:33 [2] #mmap#1(::Bool, ::Bool, ::Function, ::Mmap.Anonymous, ::Type>, ::Tuple, ::Int64) at C:\Users\RTX2080\AppData\Local\Julia-1.1.0\share\julia\stdlib\v1.1\Mmap\src\Mmap.jl:218 [3] #mmap at .\none:0 [inlined] [4] #mmap#14 at C:\Users\RTX2080\AppData\Local\Julia-1.1.0\share\julia\stdlib\v1.1\Mmap\src\Mmap.jl:251 [inlined] [5] mmap at C:\Users\RTX2080\AppData\Local\Julia-1.1.0\share\julia\stdlib\v1.1\Mmap\src\Mmap.jl:251 [inlined] [6] #File#20(::Int64, ::Bool, ::Int64, ::Nothing, ::Int64, ::Int64, ::Bool, ::Nothing, ::Bool, ::Array, ::String, ::Char, ::Bool, ::Char, ::Nothing, ::Nothing, ::Char, ::Nothing, ::UInt8, ::Nothing, ::Nothing, ::Nothing, ::Nothing, ::Dict, ::Bool, ::Float64, ::Bool, ::Bool, ::Bool, ::Bool, ::Nothing, ::Type, ::String) at C:\Users\RTX2080\.julia\packages\CSV\2IO2Z\src\CSV.jl:227 [7] (::getfield(Core, Symbol("#kw#Type")))(::NamedTuple<(:delim, :header),Tuple>, ::Type , ::String) at .\none:0 [8] #read#63 at C:\Users\RTX2080\.julia\packages\CSV\2IO2Z\src\CSV.jl:559 [inlined] [9] (::getfield(CSV, Symbol("#kw##read")))(::NamedTuple<(:delim, :header),Tuple>, ::typeof(CSV.read), ::String) at .\none:0 [10] top-level scope at util.jl:156

The text was updated successfully, but these errors were encountered:

Contributor
xiaodaigh commented May 6, 2019

Windows 10 Pro. 64G RAM. Julia 1.1 CSV.jl 0.5

quinnj commented May 6, 2019

Hmmm, this might be a windows issue then, I’ll have to dig out my windows box tonight and test some things. That mmap error looks suspicious saying it’s an error, but operation completed successfully.

For reference, data.table fread was unable to read a file this big (at least on my machine, it failed after 20 seconds with a «vector memory exhausted» error), TableReader.jl took 1711 seconds, and I had to stop TextParse.jl after about an hour and it hadn’t finished yet (but was slowing down my system as to be unuseable).

quinnj commented May 7, 2019

Ok, I can reproduce the error on my windows machine w/ 16gb ram. Playing around w/ just mmap, I can’t seem to do an anonymous mmap over about 10.5gb, and if I allocate several, I can’t even get that much. This stack overflow sounds a lot like what we’re seeing, but doesn’t give an immediate answer. Do we try to create a file full of null bytes the size that we need and then mmap that? That’s annoying because we’re actually taking up disk space w/ that temporary file and hopefully can manage to delete it when we’re done. Maybe there’s some way we can allocate smaller chunks up to what we need and concat them together somehow? That seems annoying, but more possible. I’ll keep researching this.

quinnj added a commit that referenced this issue May 7, 2019
quinnj added a commit that referenced this issue May 7, 2019

* If the source isn't an IO, let's mmap to a new mmapped buffer; it's a lot faster for large files than going through normal IO. Helps a little for #424 * Only define our grisu method for our own Floats. This was inadvertently working on Dec64, which isn't valid

quinnj commented May 7, 2019

I merged #426 which might help a little, but probably isn’t a full solution to this issue. I wonder if a 64gb ram system would work now though; if anyone wants to try #master branch, that’d be great. Otherwise, I’ll keep noodling on the possibility of how to do a chunked array approach.

quinnj mentioned this issue May 18, 2019
Contributor
xiaodaigh commented May 22, 2019
zjpi closed this as completed May 22, 2019
quinnj commented May 22, 2019

Oh good! I can still generate scenarios where a memory-constrained windows box will fail, but I’m glad that #426 at least helps when the machine has sufficient RAM.

jaakkor2 mentioned this issue May 24, 2019
Contributor
xiaodaigh commented Jun 23, 2019

The problem is now back. The first time I run it, it works fine, the second time it errors

@time a = CSV.read(path; header=0, limit=3) @time a = CSV.read(path; header=0, limit=3)

julia> @time a = CSV.read(path; header=0, limit=3) ERROR: could not create file mapping: The operation completed successfully. Stacktrace: [1] error(::String) at .\error.jl:33 [2] #mmap#1(::Bool, ::Bool, ::Function, ::Mmap.Anonymous, ::Type>, ::Tuple, ::Int64) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\Mmap\src\Mmap.jl:218 [3] #mmap#14 at .\none:0 [inlined] [4] mmap at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\Mmap\src\Mmap.jl:251 [inlined] [5] getsource(::String, ::Bool) at C:\Users\RTX2080\.julia\packages\CSV\xJZKC\src\utils.jl:159 [6] file(::String, ::Int64, ::Bool, ::Int64, ::Nothing, ::Int64, ::Int64, ::Bool, ::Nothing, ::Bool, ::Bool, ::Array , ::String, ::Nothing, ::Bool, ::Char, ::Nothing, ::Nothing, ::Char, ::Nothing, ::UInt8, ::Nothing, ::Nothing, ::Nothing, ::Nothing, ::Dict , ::Bool, ::Float64, ::Bool, ::Bool, ::Bool, ::Bool, ::Nothing) at C:\Users\RTX2080\.julia\packages\CSV\xJZKC\src\CSV.jl:237 [7] #File#21 at C:\Users\RTX2080\.julia\packages\CSV\xJZKC\src\CSV.jl:160 [inlined] [8] (::getfield(Core, Symbol("#kw#Type")))(::NamedTuple<(:header, :limit),Tuple>, ::Type , ::String) at .\none:0 [9] #read#61 at C:\Users\RTX2080\.julia\packages\CSV\xJZKC\src\CSV.jl:645 [inlined] [10] (::getfield(CSV, Symbol("#kw##read")))(::NamedTuple<(:header, :limit),Tuple>, ::typeof(CSV.read), ::String) at .\none:0 [11] top-level scope at util.jl:156